Benchmarking Robots for Everyday Environments: From Lab Experiments to Real-World Operations Journal
Memmesheimer, Raphael; Overbeck, Martina; Beyer, Dominik; Kral, Björn; Bellmann, Sabine; Schneider, Sven; Zimmermann, Jan; Meer, Anna-Maria; Klicic, Medina; Roth, Simone; Straßmann, Carolin; Arntz, Alexander; Wessels, Marlene; Kraus, Johannes; Schweidler, Paul; Schnell, Tristan; Zimmermann, Christoph; Pulver, Benedikt; Stork, Wilhelm; Gersch, Martin; Behnke, Sven; Roennau, Arne
Frontiers in Robotics and AI - Field Robotics (Accepted for publication)
(2026)
Publications
Legend:
Journal
Conference
Tech Report
Preprint
Book Chapter
2026
-
-
RimA: Roboter im Alltag - Benchmarking von Assistenzrobotern. Förderlinie Roboter für Assistenzfunktionen: Interaktion in der Praxis (RA3) - Transferzentrum RimA - Roboter im Alltag. Sachbericht zum Verwendungsnachweis Tech Report
Memmesheimer, Raphael; Behnke, Sven
Hannover: Technische Informationsbibliothek (2026)
[DOI] -
Learn2Grasp: Learning Human-Like Interactive Grasping Based on Visual and Haptic Feedback. Sachbericht zum Verwendungsnachweis; Förderprogramm: Deutsch-französische Projekte zum Thema Künstliche Intelligenz Tech Report
Memmesheimer, Raphael; Behnke, Sven
Hannover: Technische Informationsbibliothek (2026)
[DOI] -
Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation Conference
Tutevych*, Vitalii; Memmesheimer*, Raphael; Eichler, Luca; Pavlichenko, Dmytro; Schilke, Fynn; Krudewig, Rodja; Behnke, Sven
RoboCup 2026: Robot World Cup (Accepted for publication) (2026) Best Paper Award, RoboCup Symposium 2026
[arxiv] [github] -
Adaptive Domestic Service Robotics through Foundation Models for Perception, Interaction, and Action Conference
Memmesheimer, Raphael; Pavlichenko, Dmytro; Kruzhkov, Evgenii; Bode, Jonas; Schilke, Fynn; Tutevych, Vitalii; Lenz, Christian; Schreiber, Michael; Behnke, Sven
Proceedings of the 2nd German Robotics Conference (GRC) (Accepted for presentation) (2026)
-
-
EL3DD: Extended Latent 3D Diffusion for Language Conditioned Multitask Manipulation Conference
Bode, Jonas; Memmesheimer, Raphael; Behnke, Sven
Accepted for European Robotics Forum (ERF), Stavanger, Norway, March 2026 (2026)
[arxiv] -
2025
-
Integration of the TIAGo Robot into Isaac Sim with Mecanum Drive Modeling and Learned S-Curve Velocity Profiles Conference
Schönbach, Vincent; Wiedemann, Marvin; Memmesheimer, Raphael; Mosbach, Malte; Behnke, Sven
2025 IEEE 21st International Conference on Automation Science and Engineering (CASE) IEEE (2025)
[DOI] [arxiv] [github] -
-
Anticipating Human Behavior for Safe and Efficient Collaborative Mobile Manipulation Conference
Bultmann, Simon; Memmesheimer, Raphael; Nogga, Jan; Hau, Julian; Behnke, Sven
Proceedings of the 1st German Robotics Conference (GRC) (Accepted for presentation) (2025)
-
Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors Conference
Seliunina, Svetlana; Otelepko, Artem; Memmesheimer, Raphael; Behnke, Sven
IEEE/SICE International Symposium on System Integration, SII 2025, Munich, Germany, January 21-24, 2025 IEEE (2025)
[DOI] [arxiv] [github] -
NimbRo Wins ANA Avatar XPRIZE Immersive Telepresence Competition: Human-Centric Evaluation and Lessons Learned Journal
Lenz, Christian; Schwarz, Max; Rochow, Andre; Pätzold, Bastian; Memmesheimer, Raphael; Schreiber, Michael; Behnke, Sven
Int. J. Soc. Robotics (2025)
[DOI] [video] [project_page] [arxiv]
2024
-
RoboCup@Home 2024 OPL Winner NimbRo: Anthropomorphic Service Robots Using Foundation Models for Perception and Planning Conference
Memmesheimer, Raphael; Nogga, Jan; Pätzold, Bastian; Kruzhkov, Evgenii; Bultmann, Simon; Schreiber, Michael; Bode, Jonas; Karacora, Bertan; Park, Juhui; Savinykh, Alena; Behnke, Sven
RoboCup 2024: Robot World Cup XXVII Springer (2024)
[DOI] [project_page] [arxiv] -
A Comparison of Prompt Engineering Techniques for Task Planning and Execution in Service Robotics Conference
Bode, Jonas; Pätzold, Bastian; Memmesheimer, Raphael; Behnke, Sven
23rd IEEE-RAS International Conference on Humanoid Robots, Humanoids 2024, Nancy, France, November 22-24, 2024 IEEE (2024)
[DOI] [arxiv] [github] -
-
Cleaning Robots in Public Spaces: A Survey and Proposal for Benchmarking Based on Stakeholders Interviews Conference
Memmesheimer, Raphael; Overbeck, Martina; Kral, Björn; Steffen, Lea; Behnke, Sven; Gersch, Martin; Roennau, Arne
RoboCup 2024: Robot World Cup XXVII Springer (2024)
[DOI] [arxiv] [project_page]
2023
-
Audio-Based Roughness Sensing and Tactile Feedback for Haptic Perception in Telepresence Conference
Pätzold, Bastian; Rochow, Andre; Schreiber, Michael; Memmesheimer, Raphael; Lenz, Christian; Schwarz, Max; Behnke, Sven
IEEE International Conference on Systems, Man, and Cybernetics, SMC 2023, Honolulu, Oahu, HI, USA, October 1-4, 2023 IEEE (2023)
[DOI] [arxiv] -
Robust Immersive Telepresence and Mobile Telemanipulation: NimbRo wins ANA Avatar XPRIZE Finals Conference
Schwarz, Max; Lenz, Christian; Memmesheimer, Raphael; Pätzold, Bastian; Rochow, Andre; Schreiber, Michael; Behnke, Sven
22nd IEEE-RAS International Conference on Humanoid Robots, Humanoids 2023, Austin, TX, USA, December 12-14, 2023 IEEE (2023)
[DOI] [video] [arxiv] [project_page] -
External Camera-Based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors Conference
Bultmann, Simon; Memmesheimer, Raphael; Behnke, Sven
IEEE International Conference on Robotics and Automation, ICRA 2023, London, UK, May 29 - June 2, 2023 IEEE (2023)
[video][arxiv] [DOI] -
RoboCup 2022 AdultSize Winner NimbRo: Upgraded Perception, Capture Steps Gait and Phase-Based In-Walk Kicks Conference
Pavlichenko, Dmytro; Ficht, Grzegorz; Amini, Arash; Hosseini, Mojtaba; Memmesheimer, Raphael; Villar-Corrales, Angel; Schulz, Stefan M.; Missura, Marcell; Bennewitz, Maren; Behnke, Sven
RoboCup 2022: Robot World Cup XXV [Bangkok, Thailand, July 11-17, 2022] Springer (2023)
[video][arxiv] [DOI] -
On the recognition of human activities and the evaluation of its imitation by robotic systems
Memmesheimer, Raphael
(2023)
PhD thesis [url] -
Reduced Skeleton Representation for Action Recognition on Graph Convolutional Neural Networks Conference
Germann, Ida; Memmesheimer, Raphael; Paulus, Dietrich
IEEE/SICE International Symposium on System Integration, SII 2023, Atlanta, GA, USA, January 17-20, 2023 IEEE (2023)
[DOI] -
Classification of pathological and healthy individuals for computer-aided physical rehabilitation Conference
Kramer, Ivanna; Memmesheimer, Raphael; Paulus, Dietrich
IEEE/SICE International Symposium on System Integration, SII 2023, Atlanta, GA, USA, January 17-20, 2023 IEEE (2023)
[DOI]
2022
-
Data augmentation for training a neural network for image reconstruction in MPI Conference
von Gladiss, Anselm; Kramer, Ivanna; Theisen, Nick; Memmesheimer, Raphael; Bakenecker, Anna C.; Buzug, Thorsten M.; Paulus, Dietrich
International Workshop on Magnetic Particle Imaging (2022)
[DOI] -
Reconstruction of 1D Images with a Neural Network for Magnetic Particle Imaging Book Chapter
von Gladiss, Anselm; Memmesheimer, Raphael; Theisen, Nick; Bakenecker, Anna C.; Buzug, Thorsten M.; Paulus, Dietrich
Bildverarbeitung für die Medizin 2022 - Proceedings, German Workshop on Medical Image Computing, Heidelberg, June 26-28, 2022 Springer (2022)
[DOI] -
Skeleton-DML: Deep Metric Learning for Skeleton-Based One-Shot Action Recognition Conference
Memmesheimer, Raphael; Häring, Simon; Theisen, Nick; Paulus, Dietrich
IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022 IEEE (2022)
[arxiv][video][github] [DOI]
2021
-
Fusion-GCN: Multimodal Action Recognition Using Graph Convolutional Networks Conference
Duhme, Michael; Memmesheimer, Raphael; Paulus, Dietrich
Pattern Recognition - 43rd DAGM German Conference, DAGM GCPR 2021, Bonn, Germany, September 28 - October 1, 2021, Proceedings Springer (2021)[arxiv][video][github][presentation-video] [DOI]Abstract:
In this paper we present Fusion-GCN, an approach for multimodal action recognition using Graph Convolutional Network (GCNs). Action recognition methods based around Graph Convolutional Network (GCNs) recently yielded state-of-the-art performance for skeleton-based action recognition. With Fusion-GCN, we propose to integrate various sensor data modalities into a graph that is trained using a GCN model for multi-modal action recognition. Additional sensor measurements are incorporated into the graph representation either on a channel dimension (introducing additional node attributes) or spatial dimension (introducing new nodes). Fusion-GCN was evaluated on two publicly available datasets, the UTD-MHAD- and MMACT datasets, and demonstrates flexible fusion of RGB sequences, inertial measurements and skeleton sequences. Our approach gets comparable results on the UTD-MHAD dataset and improves the baseline on the large-scale MMACT dataset by a significant margin of up to 12.37% (F1-Measure) with the fusion of skeleton estimates and accelerometer measurements. -
Action Segmentation on Representations of Skeleton Sequences Using Transformer Networks Conference
Häring, Simon; Memmesheimer, Raphael; Paulus, Dietrich
2021 IEEE International Conference on Image Processing, ICIP 2021, Anchorage, AK, USA, September 19-22, 2021 IEEE (2021)[video] [DOI]Abstract:
We propose an approach for action segmentation by representing motions as images. A transformer object detection network is used to segment the sequences from the representation images. We examine different encoding approaches, normalization strategies and skeleton joint orders in an extensive experiment study. Our approach is evaluated on skeleton sequences from the PKU-MMD dataset. We successfully apply transformer networks for action segmentation on skeleton sequences. Our proposed approach achieves high class accuracies, while start and end-time estimation of the action segments are subject to further improvement. -
Customer Interaction of a Future Convenience Store with a Mobile Manipulation Service Robot Conference
Kramer, Ivanna; Memmesheimer, Raphael; Paulus, Dietrich
2021 IEEE International Conference on Omni-Layer Intelligent Systems, COINS 2021, Barcelona, Spain, August 23-25, 2021 IEEE (2021)[DOI]Abstract:
We present an approach on how to integrate mobile manipulation service robots to support customer interaction of the future convenience store. Customers are identified by their gestures and served e.g., by carrying a shopping bag, guiding the customer to shelves of interest and bringing coffee. An integration into a widely spread telecommunication application is proposed for invoicing payment and further serves as an interface to the customer. Statistics and store layouts can be requested in the application as well as an extendable list of services provided by the robot. Further, a smart shopping bag is proposed which recognizes products stored into it by their barcode. This bag handles the invoicing and allows communication to i.e. the messenger app integration. Furthermore, our proposed system gives hints in how the customers can be profiled by their encoded faces as observed by the robot. We also warn about the possibilities of creating customer profiles without being transparent for the customers. The presented task was ranked 3rd for the Future Convenience Store Challenge at the World Robot Summit 2018. -
Mobile Manipulation Hackathon: Moving into Real World Applications Journal
Roa, Máximo A.; Dogar, Mehmet Remzi; Pagès, Jordi; Vivas, Carlos; Morales, Antonio; Correll, Nikolaus; Görner, Michael; Rosell, Jan; Foix, Sergi; Memmesheimer, Raphael; Ferro, Francesco
IEEE Robotics Autom. Mag. (2021)[DOI]Abstract:
The Mobile Manipulation Hackathon was held in late 2018 during the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) to showcase the latest applications of wheeled robotic manipulators. The challenge had an open format, where teams developed an application using simulation tools and integrated it into a robotic platform. This article presents the competition and analyzes the results, with information gathered during the event and from a survey circulated among the finalist teams. We provide an overview of the mobile manipulation field, identify key areas required for further development to facilitate the implementation of mobile manipulators in real applications, and discuss ideas about how to structure future hackathon-style competitions to enhance their impact on the scientific and industrial communities. -
Next-Best-View Estimation based on Deep Reinforcement Learning for Active Object Classification Preprint
Korbach, Christian; Solbach, Markus D; Memmesheimer, Raphael; Paulus, Dietrich; Tsotsos, John K
arXiv preprint arXiv:2110.06766 (2021)
2020
-
SL-DML: Signal Level Deep Metric Learning for Multimodal One-Shot Action Recognition Conference
Memmesheimer, Raphael; Theisen, Nick; Paulus, Dietrich
25th International Conference on Pattern Recognition, ICPR 2020, Virtual Event / Milan, Italy, January 10-15, 2021 IEEE (2020)[arxiv][video] [DOI]Abstract:
Recognizing an activity with a single reference sample using metric learning approaches is a promising research field. The majority of few-shot methods focus on object recognition or face-identification. We propose a metric learning approach to reduce the action recognition problem to a nearest neighbor search in embedding space. We encode signals into images and extract features using a deep residual CNN. Using triplet loss, we learn a feature embedding. The resulting encoder transforms features into an embedding space in which closer distances encode similar actions while higher distances encode different actions. Our approach is based on a signal level formulation and remains flexible across a variety of modalities. It further outperforms the baseline on the large scale NTU RGB+D 120 dataset for the One-Shot action recognition protocol by 5.6%. With just 60% of the training data, our approach still outperforms the baseline approach by 3.7%. With 40% of the training data, our approach performs comparably well to the second follow up. Further, we show that our approach generalizes well in experiments on the UTD-MHAD dataset for inertial, skeleton and fused data and the Simitate dataset for motion capturing data. Furthermore, our inter-joint and inter-sensor experiments suggest good capabilities on previously unseen setups. -
Gimme Signals: Discriminative signal encoding for multimodal activity recognition Conference
Memmesheimer, Raphael; Theisen, Nick; Paulus, Dietrich
IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020, Las Vegas, NV, USA, October 24, 2020 - January 24, 2021 IEEE (2020)[arxiv][video] [DOI]Abstract:
We present a simple, yet effective and flexible method for action recognition supporting multiple sensor modalities. Multivariate signal sequences are encoded in an image and are then classified using a recently proposed EfficientNet CNN architecture. Our focus was to find an approach that generalizes well across different sensor modalities without specific adaptions while still achieving good results. We apply our method to 4 action recognition datasets containing skeleton sequences, inertial and motion capturing measurements as well as \wifi fingerprints that range up to 120 action classes. Our method defines the current best CNN-based approach on the NTU RGB+D 120 dataset, lifts the state of the art on the ARIL Wi-Fi dataset by +6.78%, improves the UTD-MHAD inertial baseline by +14.4%, the UTD-MHAD skeleton baseline by 1.13% and achieves 96.11% on the Simitate motion capturing data (80/20 split). We further demonstrate experiments on both, modality fusion on a signal level and signal reduction to prevent the representation from overloading. -
Robotic Imitation by Markerless Visual Observation and Semantic Associations Conference
Memmesheimer, Raphael; Kramer, Ivanna; Seib, Viktor; Theisen, Nick; Paulus, Dietrich
2020 IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2020, Ponta Delgada, Portugal, April 15-17, 2020 IEEE (2020)[DOI]Abstract:
In this paper we present an approach for learning to imitate human behavior on a semantic level by markerless visual observation. We analyze a set of spatial constraints on human pose data extracted using convolutional pose machines and object information extracted from 2D image sequences. A scene analysis, based on an ontology of objects and affordances, is combined with continuous human pose estimation and spatial object relations. Using a set of constraints we associate the observed human actions with a set of executable robot commands. We demonstrate our approach in a kitchen task, where the robot learns to prepare a meal.
2019
-
Simitate: A Hybrid Imitation Learning Benchmark Conference
Memmesheimer, Raphael; Kramer, Ivanna; Seib, Viktor; Paulus, Dietrich
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019, Macau, SAR, China, November 3-8, 2019 IEEE (2019)[arxiv][video][project-page] [DOI]Abstract:
We present Simitate, a hybrid benchmarking suite targeting the evaluation of approaches for imitation learning. A dataset containing 1938 sequences where humans perform daily activities in a realistic environment is presented. The dataset is strongly coupled with an integration into a simulator. RGB and depth streams with a resolution of $960 times 540$ at 30Hz and accurate ground truth poses for the demonstrator’s hand, as well as the object in 6 DOF at 120Hz are provided. Along with our dataset we provide the 3D model of the used environment, labeled object images and pre-trained models. A benchmarking suite that aims at fostering comparability and reproducibility supports the development of imitation learning approaches. Further, we propose and integrate evaluation metrics on assessing the quality of effect and trajectory of the imitation performed in simulation. Simitate is available on our project website: https://agas.uni-koblenz.de/simitate/. -
Scratchy: A Lightweight Modular Autonomous Robot for Robotic Competitions Conference
Memmesheimer, Raphael; Kuhlmann, Isabelle; Mints, Mark; Schmidt, Patrik; Korbach, Christian; Germann, Ida; Paulus, Dietrich
2019 IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2019, Porto, Portugal, April 24-26, 2019 IEEE (2019)[arxiv][github] [DOI]Abstract:
We present Scratchy-a modular, lightweight robot built for low budget competition attendances. Its base is mainly built with standard 4040 aluminium profiles and the robot is driven by four mecanum wheels on brushless DC motors. In combination with a laser range finder we use estimated odometry - which is calculated by encoders - for creating maps using a particle filter. A RGB-D camera is utilized for object detection and pose estimation. Additionally, there is the option to use a 6-DOF arm to grip objects from an estimated pose or generally for manipulation tasks. The robot can be assembled in less than one hour and fits into two pieces of hand luggage or one bigger suitcase. Therefore, it provides a huge advantage for student teams that participate in robot competitions like the European Robotics League or RoboCup. Thus, this keeps the funding required for participation, which is often a big hurdle for student teams to overcome, low. The software and additional hardware descriptions are available under: https://github.com/homer-robotics/scratchy. -
Adaptive Learning Methods for Autonomous Mobile Manipulation in RoboCup@Home Conference
Memmesheimer, Raphael; Seib, Viktor; Evers, Tobias; Müller, Daniel; Paulus, Dietrich
RoboCup 2019: Robot World Cup XXIII [Sydney, NSW, Australia, July 8, 2019] Springer (2019)[DOI]Abstract:
Team homer@UniKoblenz has become an integral part of the RoboCup@Home community. As such we would like to share our experience gained during the competitions with new teams. In this paper we describe our approaches with a special focus on our demonstration of this year’s finals. This includes semantic exploration, adaptive programming by demonstration and touch enforcing manipulation. We believe that these demonstrations have a potential to influence the design of future RoboCup@Home tasks. We also present our current research efforts in benchmarking imitation learning tasks, gesture recognition and a low cost autonomous robot platform. Our software can be found on GitHub at https://github.com/homer- robotics. -
Evaluation Of Physical Therapy Through Analysis Of Depth Images Conference
Kramer, Ivanna; Schmidt, Niko; Memmesheimer, Raphael; Paulus, Dietrich
28th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2019, New Delhi, India, October 14-18, 2019 IEEE (2019)[DOI]Abstract:
The support through robots in orthopaedic rehabilitation is an opportunity to relieve physiotherapists. However, to be able to provide a control in the robot-patient cooperation in the therapy process a certain standard in interpreting the exercise has to be established. In this paper we present an evaluation approach of the health subject performance in a tibiofemoral rehabilitation on the example of squat exercises. The proposed method utilizes only depth images for the performance evaluation and any human- robot interaction system for the performance correction. Thus, this method can be easily applied to a mobile service robot in the robot-aided physical therapy. The patient is observed while performing the exercise and the motion is evaluated and segmented using Motion History Images. Concrete, depth images are used to monitor local points of interest on the performer during the exercise. The proposed approach was evaluated on custom image sequences with a multitude of varying subjects and shows the suitable performance for assisting in the correctness of the exercise execution. -
Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping Conference
Schneider, Pascal; Memmesheimer, Raphael; Kramer, Ivanna; Paulus, Dietrich
RoboCup 2019: Robot World Cup XXIII [Sydney, NSW, Australia, July 8, 2019] Springer (2019)[arxiv][github] [DOI]Abstract:
Gesture recognition opens up new ways for humans to intuitively interact with machines. Especially for service robots, gestures can be a valuable addition to the means of communication to, for example, draw the robot’s attention to someone or something. Extracting a gesture from video data and classifying it is a challenging task and a variety of approaches have been proposed throughout the years. This paper presents a method for gesture recognition in RGB videos using OpenPose to extract the pose of a person and Dynamic Time Warping (DTW) in conjunction with One-Nearest- Neighbor (1NN) for time-series classification. The main features of this approach are the independence of any specific hardware and high flexibility, because new gestures can be added to the classifier by adding only a few examples of it. We utilize the robustness of the Deep Learning-based OpenPose framework while avoiding the data-intensive task of training a neural network ourselves. We demonstrate the classification performance of our method using a public dataset.
2018
-
homer@UniKoblenz: Winning Team of the RoboCup@Home Open Platform League 2018 Conference
Memmesheimer, Raphael; Mykhalchyshyna, Ivanna; Seib, Viktor; Evers, Tobias; Paulus, Dietrich
RoboCup 2018: Robot World Cup XXII [Montreal, QC, Canada, June 18-22, 2018] Springer (2018)[DOI]Abstract:
We won this year’s RoboCup@Home track in the Open Platform League in Montreal (Canada). The approaches as used for the competition are briefly described in this paper. The robotic hardware of our custom built robot Lisa and the PAL Robotics TIAGo, both running the same methods, are presented. New approaches for object recognition, especially the preprocessed segment augmentation, effort based gripping, gesture recognition and approaches for visual imitation learning based on continuous spatial observations between a demonstrator and the interacting objects are presented. Further, we present the current state of research of our Imitation Learning approaches, where we propose a hybrid benchmark and methods for bootstrapping actions. Furthermore, our research on point cloud based object recognition is presented. -
Gesture Recognition On Human Pose Features Of Single Images Conference
Memmesheimer, Raphael; Mykhalchyshyna, Ivanna; Paulus, Dietrich
9th IEEE International Conference on Intelligent Systems, IS 2018, Funchal, Madeira, Portugal, September 25-27, 2018 IEEE (2018)[DOI]Abstract:
Enabling robots to read intentions of humans i.e. by detecting a waving person in a restaurant or by pointing to something in a supermarket gives a huge set of possibilities for human robot interactions. Gesture recognition is a challenging problem because of the complexity and the wide variety of human gestures. In this paper, we propose a method for multi-person gesture classification for five gestures and one neutral body posture based on human pose features extracted on 2D images. Comparison between supervised machine learning methods for gesture classification are given. The results showed, that the proposed approach achieves good results on our own validation dataset and generalizes well on a public dataset. -
RoboCup@Home: Summarizing achievements in over eleven years of competition Conference
Matamoros, Mauricio; Seib, Viktor; Memmesheimer, Raphael; Paulus, Dietrich
2018 IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2018, Torres Vedras, Portugal, April 25-27, 2018 IEEE (2018)[DOI]Abstract:
Scientific competitions are important in robotics because they foster knowledge exchange and allow teams to test their research in unstandardized scenarios and compare result. In the field of service robotics its role becomes crucial. Competitions like RoboCup@Home bring robots to people, a fundamental step to integrate them into society. In this paper we summarize and discuss the differences between the achievements claimed by teams in their team description papers, and the results observed during the competition1 from a qualitative perspective. We conclude with a set of important challenges to be conquered first in order to take robots to people's homes. We believe that competitions are also an excellent opportunity to collect data of direct and unbiased interactions for further research.
2017
-
homer@UniKoblenz: Winning Team of the RoboCup@Home Open Platform League 2017 Conference
Memmesheimer, Raphael; Seib, Viktor; Paulus, Dietrich
RoboCup 2017: Robot World Cup XXI [Nagoya, Japan, July 27-31, 2017] Springer (2017)[DOI]Abstract:
In this paper we present the approaches that we used for this year's RoboCup@Home participation in the Open Platform League. A special focus was put on team collaboration by handing over objects between two robots of different teams that were not connected by network. The robots communicated using natural language (speech synthesis, speech recognition), a typical human-robot “interface” that was adapted to robot-robot interaction. Furthermore, we integrated new approaches for online tracking and learning of an operator, have shown a novel approach for teaching a robot new commands by describing them using natural language and a set of previously known commands. Parameters of these commands are still interchangeable. Finally, we integrated deep neural networks for person detection and recognition, human pose estimation, gender classification and object recognition. -
Joint operator detection and tracking for person following from mobile platforms Conference
Wojke, Nicolai; Memmesheimer, Raphael; Paulus, Dietrich
20th International Conference on Information Fusion, FUSION 2017, Xi'an, China, July 10-13, 2017 IEEE (2017)[DOI]Abstract:
In this paper, we propose an integrated system to detect and track a single operator that can switch off and on when it leaves and (re-)enters the scene. Our method is based on a set-valued Bayes-optimal state estimator that integrates RGB-D detections and image-based classification to improve tracking results in severe clutter and under long-term occlusion. The classifier is trained in two stages: First, we train a deep convolutional neural network to obtain a feature representation for person re-identification. Then, we bootstrap a classifier that discriminates the operator from remaining people on the output of the state-estimator. We evaluate the approach on a publicly available multi-target tracking dataset as well as custom datasets that are specific to our problem formulation. Experimental results suggest reliable tracking accuracy in crowded scenes and robust re-detection after long-term occlusion. -
Benchmarking Speech Understanding in Service Robotics Conference
Vanzo, Andrea; Iocchi, Luca; Nardi, Daniele; Memmesheimer, Raphael; Paulus, Dietrich; Ivanovska, Iryna; Kraetzschmar, Gerhard K.
Proceedings of the 4th Italian Workshop on Artificial Intelligence and Robotics A workshop of the XVI International Conference of the Italian Association for Artificial Intelligence (AI*IA 2017), Bari, Italy, November 14-15, 2017 CEUR-WS.org (2017)
[url] [pdf]
2016
-
A ROS-based System for an Autonomous Service Robot Book Chapter
Seib, Viktor; Memmesheimer, Raphael; Paulus, Dietrich
Robot Operating System (ROS): The Complete Reference (Volume 1) Springer (2016)[DOI]Abstract:
The Active Vision Group (AGAS) has gained plenty of experience in robotics over the past years. This contribution focuses on the area of service robotics. We present several important components that are crucial for a service robot system: mapping and navigation, object recognition, speech synthesis and speech recognition. A detailed tutorial on each of these packages is given in the presented chapter. All of the presented components are published on our ROS package repository: http:// wiki.ros.org/agas-ros-pkg.
2015
-
Team Homer@UniKoblenz - Approaches and Contributions to the RoboCup@Home Competition Conference
Seib, Viktor; Manthe, Stephan; Memmesheimer, Raphael; Polster, Florian; Paulus, Dietrich
RoboCup 2015: Robot World Cup XIX [papers from the 19th Annual RoboCup International Symposium, Hefei, China, July 23, 2015] Springer (2015)[DOI]Abstract:
In this paper we present the approaches and contributions of team homer@UniKoblenz that were developed for and applied during the RoboCup@Home competitions. In particular, we highlight the different abstraction layers of our software architecture that allows for rapid application development based on the ROS actionlib. This architectural design enables us to focus on the development of new algorithms and approaches and significantly helped us in winning the RoboCup@Home competition in 2015. We further give an outlook on recently published open-source software for service robots that can be downloaded from our ROS package repository on http://wiki.ros.org/agas-ros-pkg . -
Ensemble classifier for joint object instance and category recognition on RGB-D data Conference
Seib, Viktor; Memmesheimer, Raphael; Paulus, Dietrich
2015 IEEE International Conference on Image Processing, ICIP 2015, Quebec City, QC, Canada, September 27-30, 2015 IEEE (2015)[DOI]Abstract:
Sensors for RGB-D data have gained high popularity in the computer vision community. We present an efficient ensemble classifier that combines visual and depth data and achieves higher recognition rates than the individual classifiers or a classifier exploiting visual and depth data at the same time. The presented approach was evaluated in practice on a mobile robot during the RoCKIn robotics challenge in 2014.