The future is robotics (data)
August '26
Up until a few weeks ago, I was sceptical that robotics would have a major impact on the world anytime soon. I saw robotics as much more complex than language models, and projecting the success of language models onto robotics seemed wrong: language models have an enormous dataset (the internet) to learn from for free. Robotics does not.
However, in the past few weeks, my perspective has shifted considerably, and I am now 80% certain that we'll have robots capable of doing many household chores with high accuracy (>99%) in 1-2 years. If that's achieved, there should be few reasons not to deploy them at scale in households in the US in the years thereafter [1]. And what can be done in homes should be possible to do in industry as well, at least for tasks currently done by humans, given that many industry tasks are more constrained than household tasks.
What caused my shift?
- Sunday Robotics' 99.1% success rate for folding laundry, achieved mostly through careful data scaling efforts.
- Scaling laws from human video data by Dyna.
- 1X's NEO hand dexterity videos.
- The amount of money going into the robotics domain. Although not a guarantee for success, pouring lots of money and talent into a domain will typically result in a lot of progress. The past few weeks have been a whirlwind of one robotics announcement after another [2].
Points 1 and 3 will likely merge, and we will see large-scale, in-the-wild data acquisition for signals that are useful for training fine-grained hand manipulation [3].
My high-level argument is that task-level reliability is proven given enough data, scaling laws are beginning to emerge, and because data collection can be funded by capital, capital will drive improvements.
Folding clothes generalisation
Sunday Robotics' 99.1% success rate is amazing; few labs report success rates like these on tasks that are as broad as 'folding clothes', in different homes without per-home fine-tuning. When thinking about success rates like these, think about the number of errors per trials: 90% is 1 error every 10 trials, 99% is 1 in 100. A subsequent goal might be to get to 99.9%; another 10x improvement over 99%: 1 error for every 1000 trials.
But does this 99.1% success rate for clothes folding mean that we can expect good performance on other household tasks anytime soon? Let's unpack this. The report stated the following:
- Across 9 garment types
- In unseen homes without fine-tuning on that environment
- Humans rated 'fold quality' at 4.72 out of 5 stars
- Clothes in piles, in baskets, or on the floor, different lighting and a size range from 8XL shirts down to baby clothes.
As such, this seems to be a pretty convincing result; this is not a test done in a single environment for a very singular task. But even a robust result on a single task does not mean guaranteed results on other, different tasks. For one, it is easy to see that folding clothes is simpler than many other tasks. Folding has a few unique characteristics, as Chris Paxton noted in August '25:
- It's fundamentally about predicting sequences of largely non-interactive trajectories, meaning that there's minimal force being applied at any given time.
- You need a good amount of precision, but not too much precision. It's very forgiving of small mistakes, and you can use really low-cost hardware
- It's very repeatable, and very easy to reset — practically self-resetting — making it a great test task for an autonomy stack, since you can collect a lot of data very quickly.
Let's look at a different household task: loading a dishwasher. Although it is somewhat repeatable (especially if you combine loading with unloading), it requires interactions with objects where force requirements vary (a rubber container deforms more than a glass cup). You likely need more precision, as failing in real scenarios means that something might break. You do not want to use a dishwasher loading robot that breaks 1 of every 100 cups it puts into your dishwasher [4]. So, 99% accuracy for loading a dishwasher is not enough, and the task itself is more challenging. This means that a) we will need more data and b) we will need even better performance, likely requiring even more data.
And 'more data' is a general statement; you need high-quality, diverse data at scale, which is not easy for tasks with many degrees of freedom. However, the household is a great place to collect that data (see again Chris Paxton, here).
There is more reason not to expect sudden generalisation to other tasks; DeepMind Robotics recently presented somewhat meagre results, albeit on difficult tasks, and with a very different approach:
Gemini Robotics 2 on Apollo with Sharpa hands: only unscrewing a bulb clears 90%; the other four tasks land between 32% and 44%.
Data scaling
Nevertheless, I remain optimistic about progress in robotics because we've seen early, clear, and promising signs that careful scaling will go a long way. This week, Dyna showed early signs of scaling laws between human video data and robotics performance by training models on more and more video-action data (1K, 10K, 100K, 1M hours) and measuring both zero-shot robotics performance and performance after small-scale (max 10 hours) fine-tuning.
Moreover, the ML industry generally knows how to scale data to improve performance. This has been the bread and butter of many AI teams for years, and although typically not in the physical domain, the knowledge about how to do this should transfer to some extent. Robotics labs use a variety of methods to get data; Dyna uses first-person human video data without custom sensors, Sunday uses wearable glove grippers, and others use five-fingered gloves with many sensors worn by humans. These efforts are starting to yield promising results. Add more and more funding for robotics companies on top of that, and I'm confident we'll find a way.
More generally, the statement 'we do not have the internet to train on for robotics' is too imprecise; there is a lot of available data that has some transferable knowledge, and even for LLMs, internet data was never enough in the first place. Post-training/fine-tuning has been the main work for many people for years, not just for those working on LLMs, but also for people applying ML to industrial data.
In my own work, we've been trying to catch identity fraud by detecting fraudulent documents and deepfakes. Here, internet data (of passports and deepfakes) is not available, so you rely largely on what you see in production after starting from open-source pre-trained models. The production data you have is often not useful because it's data from genuine users; hence, anomalies are the gold you're looking for, but these anomalies are scarce. Yet, over time, you accumulate more of this data, and via careful labelling and mining, you can improve performance. On top of that, you can enrich your data with data augmentations, data from in-house data generation teams, and you separate tasks into the right subtasks to get performance at the right levels for one subtask when enough general data is lacking. Over time, with more data, you can group tasks by combining data from multiple tasks to get even better performance.
Blindly saying that much of this transfers to robotics is too naive (Dunning-Kruger); it will have unique challenges, but then others with more domain knowledge should be able to figure this out (and have been doing so, hence the multitude of improvements over the past couple of months).
So, more diverse and high-quality data scaling is coming, and more data scaling will lead to more results. As such, we can expect more 99% success rates on more difficult and high-risk tasks in the near future. It might take a bit of time, but with the amount of money pouring into the robotics industry right now, data generation timelines can be massively shortened.[5]
General purpose
As such, we can expect many more tasks to be done robustly by robots in the coming years; scaling both pre-training and fine-tuning data will likely go a long way. If that's true, we can expect quite a change in both industry and households, where a growing share of manual work can be replaced by robots.
I do not expect this to happen all at once, but more slowly: task by task. Firstly, because true generalisation is likely out of reach; earlier LLMs trained on an unthinkably large amount of data had pretty limited reasoning capabilities, and even now one can still easily find domains where they do not perform well, most likely because the required data to perform well is simply not part of the training data. True generalisation, even with a lot of data, is simply not something that we see much in ML. As a result, we need specific data for each task, and although one can expect to require less data for the th task as data from other tasks accumulates, this process will take a while.[6] Secondly, many tasks come with some edge cases that are not covered by other tasks, and hence, even if there is a reasonable amount of transfer knowledge here, some fresh task-specific data will still be required for high levels of accuracy. A trajectory similar to self-driving can be expected, where years of data accumulation slowly led to more autonomy, although still limited to specific locations. Robotics should go a bit faster given that failures are much less catastrophic than in self-driving.
Given the above, I expect a lot of progress on one of the true promises of AI:
I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes (source).
That will be a fun world.
Notes
[1] I haven't worked through potential problems with hardware costs, distribution at scale, and potential resistance against robots with cameras in people's homes, which reduces my certainty here.
[2] In fact, we have seen many more interesting demonstrations the past couple of weeks. Some of the news I liked:
- one model handling many grippers/hands
- tele-operated robots you can actually book to clean your house
- A drone with a robotic arm
[3] I expect the current trend of building hands with three fingers (or fewer) to fade over time. Evolution has given us some evidence that five fingers are useful for many human tasks; we will end up with robotic hands that have five fingers by default, but potentially more, or more specialised, fingers. If you're wondering why five fingers are useful, try holding your phone and typing anything with just three fingers.
[4] Error type is important here; breaking is a more catastrophic failure than, e.g. putting a cup right-side up into the dishwasher. Not all failures are equal; the metric that matters is catastrophic failures per session, not errors per item.
[5] Some estimates: with ~$200 per glove, with ~2000 shipped, initial costs are at $400K. The exact episode count is not public; Yahoo Finance reports 10M episodes. Assuming 30 seconds - 2 minutes per episode and $30-60 per hour for each 'memory developer', costs can be estimated at between $2.5-20M for the collection of data. That number is not so high that it seems impossible to repeat many times, or at a larger scale if needed (Sunday has raised $200M so far).
[6] In this DeepMind video, with models trained on teleoperation data, more evidence is provided that a lot of data (or more efficient algorithms) is needed to 'solve' robotics. Of course, more efficient algorithms would also be welcome, but so far, more data seems to be the most promising way to get to better performance.
Hannah Fry: Do you think it needs another revolution? Do we need another architecture or do you think that we've got enough already?
Kanishka Rao: I think we need at least one more breakthrough. Even now, these robots take a lot of data to learn these tasks. So we need a breakthrough where they can learn more efficiently with data.
Hannah Fry: Do you think that's the only limiting factor then? Do you think if you had a similar order of magnitude, many orders of magnitude more data like you do with language models or vision-language models, do you think that this would be sorted?
Kanishka Rao: There is one hypothesis that that's all you need. If you can collect that much robot data, then we're done. But there's still a long tail of problems to solve, they need to be safe, they have to really master the task. So there are still challenges but the core of the problem is robot data, it's physical interaction data. It's currently limited. It is not as big as the internet.