Competing in RoboCup@Home and winning the 2026 best paper award: an interview with Raphael Memmesheimer

Team NimbRo’s robot in action during the RoboCup@Home competition at RoboCup 2026. Raphael leads the team.

Raphael Memmesheimer has been successfully competing in the RoboCup@Home league winning the RoboCup@Home competition five times. He and his colleagues won the best paper award at RoboCup 2026 for their work “Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation” . We caught up with Raphael to find out more about this research, his history with RoboCup, and the challenges of the @Home competition.

Could you start by telling us how you got involved with RoboCup@Home?

I’ve actually been involved in the @Home league since I was a student, and my first time going to a World Cup for @Home was in 2013. That competition motivated me to do a PhD and infected me in a way that meant I basically aligned my academic career to it. I always return to robot competitions because this is one of the rare places where you can benchmark robots in a real environment and have a real competition. Commonly in research you tune your approaches on data sets and show that they perform in a one-shot video. In robot competitions, you basically have to roll out your robot, it enters the arena, and then it operates autonomously on its own. It is always interesting to see how new developments make the robots perform better.

You and your co-authors won the best paper award at the RoboCup symposium this year. Congratulations! Could you give us a quick introduction to the paper?

The motivation for this work is that we only have two preparation days at the RoboCup competitions. So we arrive there, we have to assemble our robots, and the robots need to get used to the new environment [a model apartment] that we haven’t seen before. Also, the robots don’t have a clue about anything in this apartment. And within the two preparation days, the robots need to learn the objects they are interacting with. For example, they have to recognise what a spoon is, or a specific drink.

Our paper basically supports us in very efficiently generating this data such that the robots can more quickly learn models for the objects that are involved in this competition. And the interesting thing is it’s not just a few objects that we can distinguish between; there can be up to 50 different objects that we haven’t seen before. Our approach basically optimizes the process so that we can quickly get a good vision model for the robot. We are aware of open-vocabulary approaches but found them to have issues distinguishing between objects of a category like drinks.

Is it possible to do some pre-training to anticipate objects that might be used in the competition?

Partially that’s possible, but most objects cannot really be anticipated. We know that there are drinks, there are snacks, that there are cutlery items in the apartment. But what cannot be anticipated is the details because objects come from different areas in the world. The organisers will usually go to a shopping mall and just get local groceries there. Those objects are then distributed in the apartment. Part of the object set comes from the publicly available YCB Dataset , for those we can gather data beforehand, and of course pre-training with large general object sets.

Could you talk about your methodology?

We use a multi-stage approach. Firstly, we segment, such that we learn what an object actually is on a segmentation level. We look at all the objects that robots saw in the past editions of these competitions, and we train one model that just says, “oh, there is something which might be an object” . And then we get reference images of the actual objects and we semi-supervised or semi-automatically label those objects. The idea is that we combine identifying something as an object with a reference of what could actually be there. That allows us to simplify the labeling approach. Around 60% of the data is automatically labeled based on our approach. That would take a lot of hours by human annotation.

Labeled example images from different competition venues. Figure from “Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation” .

Have you been able to use the method already in RoboCup competitions?

Yes, we used it in Brazil for RoboCup 2025 and at RoboCup GermanOpen for the local RoboCup competitions. The method is continuously evolving. Now that we have this very good vision model, it means that we save a lot of time during the two preparation days and we can concentrate more on testing and achieving the actual tasks instead of just focusing on the vision element. One side note: nowadays there are promptable object detection or segmentation methods, but we found that they don’t scale yet to the number of classes that are involved in this competition. They work fine with 5 to 10 different classes, but we had issues distinguishing all 50 classes just by prompting them.

In this year’s World Cup (in Korea) we used an evolved approach where we used videos to generate even more data.

Team NimbRo’s robot picking laundry from the washing machine.

How did your team get on in the competition this time?

It went well. We had issues, as always during competitions, but we ended up in 3rd place. There was a new “Doing Laundry” task introduced this year. The robot had to carry a laundry basket from a washing machine to a folding table and open the washing machine. It had to detect the washing machine door, then find the specific point to grasp the handle to open the machine door. Then it had to pull out t-shirts and fold them.

Our robot performed the best in this task. It was able to open the washing machine and start folding the laundry. And this was possible because we used these semi-automatic labeling approaches.

What were the other tasks that teams needed to carry out?

The first big hurdle at the competition is to make the robot pass the initial inspection. Robots must satisfy certain safety and…

News is gathered automatically from public robotics & AI feeds on a schedule.