Richard Sutton

Their words

then we're entering the age of design where because our AIs are designed our our our all of our physical objects are designed our buildings are designed our technology is designed and we're we're designing now uh AIs things that can be intelligent themselves and that are themselves capable of design

Their words

So I do think succession to digital or digital intelligence or augmented humans is inevitable.

Their words

But it will not be that easy, as easy as you're imagining because uh that you can lose your mind this way. If you you pull in something from the outside and build it into your into your inner thinking, uh, it could take over you. It could change you.

Their words

The bitter lesson. Oh, who cares about that? That's that's an empirical observation about a particular period in history. 70 years in history no longer doesn't necessarily have to apply the next 70 years.

Their words

I I really view myself as a classicist rather than as a contrarian. I go to what what the larger community of of thinkers about the mind have always thought.

Their words

We don't we don't really know what information they had prior. We are we have to guess because they've been fed so much. This is one reason why they're not a good way to do science. Uh it's just so uncontrolled, so unknown.

Their words

so we know deep learning is really bad at this for example we know that if you train on some new thing it will often catastrophically interfere with all the old things that you that you knew

Their words

Well, there's nothing in them which will cause it to generalize. Well, the gradient descent will cause them to find a solution to the problems they've seen. And if there's only one way to solve them, you know, they they'll do it. But there are many ways to solve it. Some which generalize well, some which generalize poorly. There's nothing in them in the algorithms that will cause them to generalize well.

Their words

We don't have any methods that are good at that. What we have are people um try different things and they they settle on something that that uh a representation that that transfers well or they generalize as well. But we have no we don't have any automated techniques to promote. we have very few automated techniques to promote transfer and they're not none of them are used in in modern deep learning.

Their words

when you learn to play chess you have the grand the long-term goal is winning the game and yet you you can't you um you want to be able to learn from shorter term things like you know taking the your opponent's pieces um and so you do that by having a value function which predicts the long-term outcome

Their words

You can't have one child learn grow up and and learn about the world and then and then every new child has to repeat that process. Whereas with AIS, with a digital intelligence, you could hope to do it once and then copy it into the next one as a starting place.

Their words

if we understood a squirrel I think we'd have a we'd be almost all the way there to understanding human intelligence. The the language part is just a a small veneer on the surface.

Their words

Why are you trying to distinguish humans? Humans are animals. What we have in common is more interesting. What distinguishes us, we should be paying less attention to.

Their words

squirrels don't go to school. Squirrels can learn all about the world. It's absolutely obvious I would say that um supervised learning doesn't happen in animals.

Their words

If you go to look about how psychologists think about learning, there's nothing like uh imitation. Maybe there are some extreme cases where humans might do that or appear to do that, but there's no basic animal learning process called imitation.

Their words

So I don't think uh learning is really about training. I think learning is about about learning. It's about an active process. The child tries things and sees what happens.

Their words

But in fact and in practice it has always turned out to be bad because people get locked into the human knowledge approach and they psychologically or you know now I'm now I'm speculating why it is but this is what has always happened.

Their words

The scalable method is you learn from experience. Um you uh you you try things, you see what you see what works. No one no one has to tell you. First of all, you have a goal. So without a goal, uh there's no sense of right or wrong or better or worse. So large language models are trying to get by without having a goal or a sense of better or worse. That's just, you know, it's exactly starting in the wrong place.

Their words

For me, having a goal is the essence of intelligence

Their words

So there's no ground truth. You can't have prior knowledge if you don't have ground truth because the prior knowledge is supposed to be a hint or an initial belief about what the truth is.

Their words

they have the ability to predict what a person would say they don't have the ability to predict what will happen

Their words

reinforcement learning is about understanding your world whereas large language models are about mimicking people doing what people say you should do. They're not about figuring out what to do.