Roadmap

Relevant Literature

League Publications

1. L. Contreras, Y. Matsusaka, T. Yamamoto, and H. Okada, "sDSPL - Towards a benchmark for general-purpose task evaluation in domestic service robots," TechRxiv preprint, 2021. [url]
2. M. Matamoros, V. Seib and D. Paulus, "Trends, Challenges and Adopted Strategies in RoboCup@Home," in IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2019, pp. 1-6. [url]
3. N. Massouh, L. Brigato, and L. Iocchi, "RoboCup@Home-Objects: Benchmarking Object Recognition for Home Robots," in oboCup 2019: Robot World Cup XXIII, pp. 397-407. [url]
4. M. Matamoros, V. Seib, R. Memmesheimer and D. Paulus, "RoboCup@Home: Summarizing achievements in over eleven years of competition," in IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2018, pp. 186-191. [url]
5. M. Matamoros, K. Harbusch, and D. Paulus "From Commands to Goal-Based Dialogs: A Roadmap to Achieve Natural Language Interaction in RoboCup@Home," in RoboCup 2018: Robot World Cup XXII, pp. 217-229. [url]
6. L. Iocchi, D. Holz, J. Ruiz-del-Solar, K. Sugiura and T. van der Zant, "RoboCup@Home: Analysis and results of evolving competitions for domestic and service robots," Artificial Intelligence, vol. 229, 2015, pp. 258-281. [url]
7. D. Holz, L. Iocchi, and T. van der Zant, "Benchmarking intelligent service robots through scientific competitions: The RoboCup@ Home approach," in AAAI Spring Symposium Series, 2013. [url]
8. T. Wisspeintner, T. van der Zant, L. Iocchi, and S. Schiffer, "RoboCup@ Home: Scientific competition and benchmarking for domestic service robots," Interaction Studies, vol. 10, no. 3, 2009, pp. 392-426. [url]

Related Articles and Survey Papers

1. I. Lee, "Service Robots: A Systematic Literature Review", Electronics, vol. 10, no. 21, e2658, 2021. [url]
   • Discusses some important challenges for service robots in general, in particular ethics, privacy, safety and security (similar to Zachiotis et al.)
2. S. Srivastava et al., "BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments," in Conference on Robot Learning (CoRL), 2021. [url]
3. M. Basiri, E. Piazza, M. Matteucci, and P. Lima, "Benchmarking Functionalities of Domestic Service Robots Through Scientific Competitions," KI-Künstliche Intelligenz, vol. 33, no. 4, pp. 357-367, 2019. [url]
   • Presents the evaluation approach used in the European Robotics League, which is split between functionality benchmarks (controlled tests of individual functionalities, such as object detection) and task benchmarks (directly comparable to our @Home tasks)
4. H. Li, S. Milani, V. Krishnamoorthy, M. Lewis, and K. Sycara, "Perceptions of Domestic Robots' Normative Behavior Across Cultures," in Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 345-351, Jan. 2019. [url]
   • Investigates the importance of a domestic robot's adaptation based on context and cultural norms
5. G. A. Zachiotis, G. Andrikopoulos, R. Gornez, K. Nakamura, and G. Nikolakopoulos, "A Survey on the Application Trends of Home Service Robotics," in Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1999-2006, 2018. [url]
   • Discusses different application areas for service robots. Lists some relevant challenges at the end (reliability, security, ethics, acceptability)
6. F. Amigoni, M. Luperto, and V. Schiaffonati, "Toward generalization of experimental results for autonomous robots," Robotics and Autonomous Systems, vol. 90, 2017, pp. 4-14. [url]
7. T. S. Tadele, T. de Vries, and S. Stramigioli, "The Safety of Domestic Robotics: A Survey of Various Safety-Related Publications," IEEE Robotics & Automation Magazine, vol. 21, no. 3, 2014, pp. 134-142. [url]
8. I. Leite, C. Martinho, and A. Paiva, "Social Robots for Long-Term Interaction: A Survey", International Journal of Social Robotics, vol. 5, 2013, pp. 291-308. [url]
9. M. J. Islam, J. Hong and J. Sattar, "Person Following by Autonomous Robots: A Categorical Overview", International Journal of Robotics Research, vol. 38, issue 14, 2019. [url]

Domestic Robot Functionalities

A useful domestic service robot needs to be equipped with a wide variety of functionalities and fulfill various usability criteria. This section makes an attempt at summarising such functionalities and criteria so that (a) our competition tasks can cover as many of those as possible and (b) the extent to which they are covered in the competition is proportional to their relevance for the community. For each functionality, we assign:

• a qualitative importance score between 1 and 5 according to the following scale:

1. might be good to have, but not universally important
2. good to have, but only somewhat important
3. somewhere in between; no clear agreement can be reached
4. quite important, would significantly increase the robot's usefulness
5. essential functionality

• a qualitative difficulty score on a scale between 1 and 5 based on the current state of (a) research solutions for the problem and (b) software solutions - this score is thus supposed to be regularly updated according to advances in the state of the art:

1. generally solved research problem for which open-source software solutions are readily available
2. generally solved problem from a research perspective, but software solutions are lacking
3. somewhere in between; no clear agreement can be reached
4. various aspects of the problem have been addressed by the community, but various open challenges remain
5. a generally open research problem

The score assignment is done jointly by the TC. The product of the importance and difficulty scores is used to calculate a competition relevance score, the idea being that the more important and difficult a functionality is, the more relevant it is to cover it in the competition.

The list of functionalities and their scores is compiled based on:

• relevant publications about service robots in general and domestic robots in particular - both from the league and from the broader research community
• answers to a survey that was sent to the robotics community at the end of 2021 (the survey results are given here)
• an initial list of functionalities that was collected by previous TC members in the context of this related issue

Domain	Functionalities	Importance (i) (1-5)	Difficulty (d) (1-5)	Relevance (r) i*d
Navigation
Manipulation
Perception
World modelling
Cognition
Human-robot interaction
Reliability and safety
Security
Ethics