Chelsea Finn

Their words

uh the we found that that leads to improvement and we saw in the shirt folding example we saw like a quantitative bump from using that sort of imagination compared to not using it. At the same time I think that the model actually performed surprisingly well without that as well.

Their words

Uh and just like how we see in language models how now a lot of time is spent actually generating data, generating synthetic data by actually running the model and having it think through things. I think a lot of the data in the future in robotics is going to be the robot attempting to do lots of tasks in lots of real world circumstances.

Their words

Uh and likewise um robots can't like watch a person doing something and then figure out how to do it themselves directly. They really need their experience on their own platform um to learn effectively.

Their words

I mean at the very least I actually think that just starting with a generalist policy and then fine-tuning it even like right off the bat uh can be really effective.

Their words

at the same time in terms of the capabilities of these models I think that we are really starting to get to the point where these models are actually useful in the real world and I think that getting to the kind of the capabilities of chat GBT I think is um yeah very much on the horizon in the next few years.

Their words

I think that the distribution channel for physical models is going to be slower uh unfortunately because you actually need a physical robot there

Their words

without metadata prompting when you add lower quality data from 80% data to 100% data the performance actually decreases which is perhaps not too surprising because you're adding lowquality data to your data mixture whereas with the metadata prompting the performance actually increases when you add that lowquality data

Their words

we find that the performance on held out tasks decreases dramatically. Whereas if we um just take out a random 20% of the data that's less diverse than the most diverse subset, the performance um only decreases a little bit. And so this suggests that actually having really diverse data plays an important role in enabling it to generalize to new tasks.

Their words

we see that the across the board the single PIO like pre-trained PIO7 model matches or outperforms the fine-tuned specialists that were developed with reinforcement learning post-training for those downstream tasks.

Their words

when you can basically bridge the concept of an avocado and a chair and show that you kind of can combine those two, it means that the model has at least some kind of conceptual understanding of what an avocado is and what a chair is to the point that it can combine them into something that exhibits both concepts at the same time.

Their words

But if you have to fine-tune a model, you actually aren't getting a general purpose model um for the things that you want it to do because you have to fine-tune it for each individual thing.

Their words

And then for longer memory, for memory that spans multiple minutes or multiple hours, we don't necessarily need video of exactly what happened in that past uh in that past history.

Their words

So, you might be surprised to hear that most state-of-the-art foundation models for robotics have no memory or no context. They're just operating on the current sensor observations, the current camera readings, uh, and predicting actions based off of that.

Their words

kind of going back to this reliability question, we took this policy and we ran it not just once, but we ran it for 13 hours straight. Uh and we basically wanted to evaluate is this policy not only good at making a latte once, but can it do so reliably to the extent that it would be needed to be useful in the real world?

Their words

and trying to fold two boxes together isn't useful data that will teach the model how to get better at the task. And so that would be kind of wasting a lot of time on the robot attempting to go down the wrong path for solving the problem. And so instead of spending a lot of time trying to do that task, what we'll do is we'll actually have a human intervene and show the robot what to do and how to recover from that situation.

Their words

Now maybe this isn't completely out of the question but this would be quite challenging uh to do and that's because the calculus is a little bit different. We're not just running compute to optimize for a use case. We're actually running the robot in the real world and using the hardware and attempting the task in the real world.

Their words

Um, and while this generally improves the reliability of the model, uh, people eventually get tired and it's hard to get really, really high reliability with a person that's manually tuning this. And so what would be even better is if the AI system itself can iterate on the scenario in which you want it to have higher reliability where it on its own automatically seeks out places where it needs more data, where it needs more supervision.

Their words

Uh and this means that they're going to be far more useful when they're operating fully autonomously. And as a result, this requires us to develop physical AI systems that make far fewer mistakes than the machine learning systems that have been deployed thus far.

Their words

And in all of these applications, the customer is making a decision based off of the recommendation of the AI model more or less. Uh, and this means that if the customer is ultimately like kind of making the decision, this means that if the system makes a mistake, um, that's okay because usually the person can kind of recognize that or or decide what to do even despite that mistake.