{"id":264049,"date":"2024-10-17T09:38:59","date_gmt":"2024-10-17T14:38:59","guid":{"rendered":"https:\/\/mathewingram.com\/work\/?p=264049"},"modified":"2025-01-30T16:42:01","modified_gmt":"2025-01-30T21:42:01","slug":"is-ai-going-to-save-us-or-kill-us-even-the-experts-dont-agree","status":"publish","type":"post","link":"https:\/\/mathewingram.com\/work\/2024\/10\/17\/is-ai-going-to-save-us-or-kill-us-even-the-experts-dont-agree\/","title":{"rendered":"Is AI going to save us or kill us? Even the experts don’t agree"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

Like many, I’ve been fascinated by the speed with which artificial intelligence has taken over the spotlight as the technology that everyone is either excited by, confused by, or terrified by (or possibly all three). Part of that, I think, has to do with the speed at which the group of things we call AI have been evolving \u2014 it’s hard to believe that the term AI was mostly restricted to academic circles as recently as 2022, when OpenAI’s ChatGPT was released in the wild<\/a>. Then came visual AI engines like DALL-E and Midjourney, which generated some hilarious photographs and video clips, like the widely-lampooned video of an AI version of Will Smith trying to eat spaghetti<\/a>, which is alternately laughable and also creepy, in a way that only AI art seems to be. ChatGPT and other AI engines based on large language models routinely generated nonsensical results \u2014 or “hallucinations,” as some call them \u2014 where they just make things up out of thin air.<\/p>\n\n\n\n

Within a matter of months, however, those same AI chatbots were producing high-quality transcriptions and summaries, and the AI photo and video engines were generating incredibly lifelike pictures of things that don’t exist, and videos of people and animals that are virtually indistinguishable from the real thing. I recently took a test that Scott Alexander of Astral Codex Ten sent to his newsletter readers<\/a>, which presented them with pictures and asked which ones were generated by AI and which by humans, and I have zero confidence that I got any of them right. ChatGPT’s various iterations, meanwhile, have not only aced the Turing test<\/a> (which determines whether an AI is able to mimic being human) but the LSAT and a number of other tests. It’s true that AI engines like Google’s have told people to do stupid thing like eat rocks<\/a>, but the speed with which their output has become almost indistinguishable from human content is staggering.<\/p>\n\n\n\n

I should mention up front that I am well aware of the controversy over where AI engines get all the information they use to generate video and photos and text \u2014 the idea that their scraping or indexing of books and news articles is theft<\/a>, and they should either pay for it or be prevented from using it. If I were an artist whose name has become a prompt for generating images that look like his work, I might think differently. But for me, the act of indexing content (as I’ve argued for the<\/a> Columbia Journalism Review) is not that different from what a search engine like Google does, which I believe should qualify as fair use under the law (and has in previous cases such as the Google Books case and the Perfect 10 case.) Whether the Supreme Court agrees with me remains to be seen, of course, but that is my belief. I’m not going to argue about that here, however, because that is a separate question from the one I’m interested in exploring right now.<\/p>\n\n\n\n

Note<\/strong>: This is a version of my Torment Nexus newsletter, which I send out via Ghost, the open-source publishing platform. You can see other issues and sign up here<\/a>.<\/em><\/p>\n\n\n\n\n\n\n\n

Note<\/strong>: In case you are a first-time reader, or you forgot that you signed up for this newsletter, this is The Torment Nexus. You can find out more about me and this newsletter in\u00a0<\/em><\/em>this post.<\/em><\/a> This newsletter survives solely on your contributions, so please sign up for a paying subscription or visit my Patreon, which you can <\/em><\/em>find here<\/em><\/a>. I also publish a daily email newsletter of odd or interesting links called When The Going Gets Weird, <\/em><\/em>which is here<\/em><\/a>.<\/em><\/em><\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Doom or utopia?<\/h2>\n\n\n\n

The question I’m interested in is whether AI is inherently bad, in the sense that it is likely to develop something approaching or even surpassing human intelligence (known as AGI or artificial general intelligence), at which point it will cause humanity harm or even exterminate us. There are already large camps of “AI doomers”<\/a> who believe that the above will almost certainly be the case, as well as those who think we should press ahead anyway, and that supersmart AI will solve all of humanity’s problems and usher us into a utopia<\/a>, a group who are often called “accelerators.” What makes this interesting, to me at least, is not that there are people who think it is bad and others who think it is great, but that even the top scientists in this field \u2014 the godfathers of modern AI \u2014 can’t seem to agree on whether their creation is going to be the biggest boon to mankind since the discovery of fire, or whether it is likely to go rogue and kill us all.<\/p>\n\n\n\n

Geoff Hinton, for example, is widely viewed as one of the founders of modern AI, since he co-developed neural networks<\/a> and other technologies such as “backpropagation<\/a>” that are used in most of the AI engines of today, including work for which he and his partner just won the Nobel Prize<\/a>. One of his grad students, Ilya Sutskever, went on to co-found what became OpenAI and has since left to start his own AI company based on developing AI safely. Hinton \u2014 who said in accepting the Nobel that he was proud one of his students (Sutskever) “fired Sam Altman” \u2014 quit working at Google<\/a> because he wanted to speak freely about the dangers of AI. He has said in interviews with the New Yorker<\/a> and others that he came to believe AI models such as ChatGPT (GPT stands for “generative pre-trained transformer) were developing human-like intelligence faster than he expected. \u201cSometimes I think it\u2019s as if aliens had landed and people haven\u2019t realized because they speak very good English,\u201d he told MIT’s Tech Review<\/a>.<\/p>\n\n\n\n

Neural networks are widely believed to be bad at learning, in part because it takes vast amounts of data and energy to train them, and they seem to be very slow at developing new skills, even simple things like adding numbers. Brains seem to pick up new ideas and skills quickly, using a fraction of the energy that neural networks require. But when it comes right down to it, Hinton isn’t convinced that humans have any built-in superiority \u2014 if anything, he thinks it’s the opposite. \u201cOur brains have 100 trillion connections,\u201d he told the MIT Tech Review<\/a>. \u201cLarge language models have up to half a trillion, a trillion at most. Yet GPT-4 knows hundreds of times more than any one person does. So maybe it\u2019s actually got a much better learning algorithm.\u201d AI critics like to say that the technology is just glorified auto-complete, he told the New Yorker<\/a>, but “in order to be good at predicting the next word, you have to understand what\u2019s being said.”<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Building the butterfly<\/h2>\n\n\n\n

Hinton says he thinks that AI engines such as GPT-4 can comprehend the meanings of words and ideas, and therefore it’s just a matter of time before they make the jump to being able to reason independently. “It\u2019s analogous to how a caterpillar turns into a butterfly,\u201d he told the New Yorker<\/a>. \u201cIn the chrysalis, you turn the caterpillar into soup\u2014and from this soup you build the butterfly.\u201d We tend to prize reason over intuition, but Hinton believes that we are more intuitive than we acknowledge. \u201cFor years, symbolic-AI people said our true nature is, we\u2019re reasoning machines,\u201d he said. \u201cI think that\u2019s just nonsense. Our true nature is, we\u2019re analogy machines, with a little bit of reasoning built on top, to notice when the analogies are giving us the wrong answers, and correct them.\u201d<\/p>\n\n\n\n

As for the so-called \u201challucinations\u201d that AI bots sometimes come out with (Hinton uses the term \u201cconfabulations,\u201d which is more correct in a psychological sense), critics argue that this shows that LLMs have no true understanding of what they say. But Hinton argues that this is a feature<\/a>, not a bug, and that hallucinating or confabulating is among the most human attributes of an AI. “People always confabulate,” he told the New Yorker \u2014 half-truths and invented details are a normal part of human conversation. “Confabulation is a signature of human memory,” he says. “These models are doing something just like people.” Hinton adds that he suspects that skepticism<\/a> of AI\u2019s potential is often motivated by an unjustified faith in human exceptionalism. In other words, there’s nothing that different or special about the human mind, which means it should be relatively simple for an advanced AI engine to duplicate \u2014 and then surpass.<\/p>\n\n\n\n

That’s Geoff Hinton’s view. But one of his fellow ground-breaking AI researchers, Yann LeCun \u2014 an NYU professor and senior researcher at Meta who has been a friend and coworker of Hinton’s for more than 40 years \u2014 believes almost the exact opposite. LeCun told the Wall Street Journal recently that warnings about the technology\u2019s potential for existential peril are “complete B.S.”<\/a> LeCun, who won the Turing Award \u2014 one of the top prizes in computer science \u2014 in 2019 thinks that today\u2019s AI models are barely even as smart as an animal, let alone a human being, and the risk that they will soon become all-powerful supercomputers is negligible at best. Hinton has said that he thinks ChatGPT has shown signs of something approaching consciousness, but LeCun says that before we get too worried about the risks of superintelligence, “we need to have a hint of a design for a system smarter than a house cat,\u201d as he put it on X<\/a>.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Thinking or just talking?<\/h2>\n\n\n\n

Today\u2019s AI models are really just predicting the next word in a text, he says, but they\u2019re so good at this that they fool us into thinking that they are intelligent. That and their vast capacity for remembering things can make it seem as though they are reasoning, LeCun told the Journal, when in fact they\u2019re merely regurgitating information<\/a> they\u2019ve been trained on. \u201cWe are used to the idea that people or entities that can express themselves, or manipulate language, are smart \u2014 but that\u2019s not true,\u201d says LeCun. \u201cYou can manipulate language and not be smart, and that\u2019s basically what LLMs are demonstrating.\u201d Part of LeCun’s skepticism stems from the fact that he believes AI researchers are going in the wrong direction, and that true intelligence and reasoning ability won’t come from simply expanding the size of an AI database or giving it more processors. He believes that AI software needs to be trained like<\/a> a baby learns, in order to develop a model of the world and how human beings operate within it, and what the rules are.<\/p>\n\n\n\n

In addition to the back-and-forth between the godfathers of AI over whether it is going to kill us all, another thing that fascinates me about the discussion is that even some AI experts don’t know how large-language models do what they do. Tech Review describes how two scientists<\/a> left one of their experiments running for days instead of hours and found that the AI had somehow figured out how to add two numbers. They found that in certain cases, LLMs could fail repeatedly to learn a task and then all of a sudden just get it, as if a lightbulb had switched on, which they called “grokking<\/a>.” This isn\u2019t how deep learning is supposed to work, and yet somehow it did. And the recipes for how to do this are still more alchemy than chemistry: \u201cWe figured out certain incantations at midnight after mixing up some ingredients,\u201d as one researcher put it.<\/p>\n\n\n\n

Perhaps it’s not that surprising that even AI experts don’t ultimately know whether it will save us or kill us. Anyone who has seen the movie Oppenheimer<\/em>, or read any of the books about the Manhattan Project (a popular thing to read at safety-conscious Anthropic AI, according to the Times<\/a>) knows that even as the scientists were developing the first atomic bomb \u2014 which was arguably as complicated than developing a neural network, if not more so \u2014 some top scientists were afraid that having radioactive material go critical inside the atomic bomb could start a fission reaction<\/a> that would ignite the earth’s atmosphere and bring an end to life as we know it. That obviously didn’t happen. Whether AI research starts a chain reaction that ends with killer robots from Cyberdyne Systems is also unknown \u2014 even by those who have devoted their lives to this kind of research. Which is somehow fascinating and frightening at the same time.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Got any thoughts or comments? Feel free to either leave them here, or post them on <\/em><\/em>Substack<\/em><\/a> or on Ghost<\/a>, or you can also reach me on <\/em><\/em>Twitter<\/em><\/a>, <\/em><\/em>Threads<\/em><\/a>, <\/em><\/em>BlueSky<\/em><\/a> or <\/em><\/em>Mastodon<\/em><\/a>. And thanks for being a reader.<\/em><\/em><\/p>\n

<\/div>","protected":false},"excerpt":{"rendered":"

Like many, I’ve been fascinated by the speed with which artificial intelligence has taken over the spotlight as the technology that everyone is either excited by, confused by, or terrified by (or possibly all three). Part of that, I think, has to do with the speed at which the group of things we call AI … <\/p>\n

Continue reading “Is AI going to save us or kill us? Even the experts don’t agree”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crsspst_to_mathewingramblogwordpresscom":true,"mf2_syndication":[],"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[20],"tags":[],"class_list":["post-264049","post","type-post","status-publish","format-standard","hentry","category-newsletters"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts\/264049","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/comments?post=264049"}],"version-history":[{"count":9,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts\/264049\/revisions"}],"predecessor-version":[{"id":270856,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/posts\/264049\/revisions\/270856"}],"wp:attachment":[{"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/media?parent=264049"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/categories?post=264049"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mathewingram.com\/work\/wp-json\/wp\/v2\/tags?post=264049"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}