What does Geoffrey Hinton believe about AGI existential risk?

I was struck by a recent article in the (often quite interesting) New Statesman.  Here is Harry Lambert interviewing Hinton:

I asked Hinton for the strongest argument against his own position [on AGI risk]. “Yann thinks its rubbish,” he replied. “It’s all a question of whether you think that when ChatGPT says something, it understands what it’s saying. I do.”

There are, he conceded, aspects of the world ChatGPT is describing that it does not understand. But he rejected LeCun’s belief that you have to “act on” the world physically in order to understand it, which current AI models cannot do. (“That’s awfully tough on astrophysicists. They can’t act on black holes.”) Hinton thinks such reasoning quickly leads you towards what he has described as a “pre-scientific concept”: consciousness, an idea he can do without. “Understanding isn’t some kind of magic internal essence. It’s an updating of what it knows.”

In that sense, he thinks ChatGPT understands just as humans do. It absorbs data and adjusts its impression of the world. But there is nothing else going on, in man or machine.

“I believe in [the philosopher Ludwig] Wittgenstein’s position, which is that there is no ‘inner theatre’.” If you are asked to imagine a picnic on a sunny day, Hinton suggested, you do not see the picnic in an inner theatre inside your head; it is something conjured by your perceptual system in response to a demand for data. “There is no mental stuff as opposed to physical stuff.” There are “only nerve fibres coming in”. All we do is react to sensory input.

The difference between us and current AIs, Hinton thinks, is the range of input.

As I interpret that passage, the claim is that AGI risk is significant because ChatGPT possesses sentient understanding of a kind not less valid than what humans have, though the current GPT understanding is monomodal only.  Furthermore, humans (and AIs) should not be understood as being conscious.  On top of that, he seems to suggest that his arguments for AGI risk rest upon the fundamental non-divergence of humans and AIs.

Now, I am not sure what is the polite way to put this, but…I think those arguments are just a teeny, tiny, teensy, trifling bit ever so slightly wrong.  As in “not entirely accurate” by at least a smidgen.

No, I am not going to go “full Bryan Caplan” on you, as I believe we are not in control through our “inner theatre,” and furthermore the nature of that “inner theatre” is not transparent to us.  But an inner theatre there is of some kind!  I am not persuaded otherwise, not even if your inner theatre is screaming “no”!

Hinton’s view oddly cuts against his own AGI risk argument.  Let’s say there were some kind of intrinsic difference between AIs and the human “inner theatre,” as my own inner theatre seems to want to insist.  Why does that make it so hard to imagine highly dangerous AI scenarios?  (And I am hardly the biggest AI doomster.)

Here is further feedback from Philip Ball:

I’m quite taken aback at some of the simplistic comments Hinton makes. He seems to feel that the only thing separating deep-learning AI from the human mind is a matter of scale. I can’t fathom this conviction that somehow all intelligence must be heading towards ours.

…”But there is nothing else going on, in man or machine.” But this is not how cognitive scientists think about human cognition.

And more at the link.  You might argue that, in considering Hinton’s claims, I am not steelmanning the AI worry arguments.  But that is exactly the point here.  We have a movement — the AGI worriers — that uses single sentence statements, backed by signatures and arguments from authority.  Hinton signed the latest letter, and in this area his name carries a lot of force.

So perhaps it is worth looking more closely at what those authorities actually believe, and whether that should impress us.