NFDI4Culture - Consortium for research data on material and immaterial cultural heritage




In the last years, we have seen a tremendous progress in the capabilities of computer systems to classify image or video clips taken from the Internet or to analyze human pose in real-time for gaming applications. These systems, however, analyze the past or in the case of real-time systems the present with a delay of a few milliseconds. For applications, where a moving system has to react or interact with humans, this is insufficient. For instance, robots collaborating with humans need not only to perceive the current situation, but they need to anticipate human actions and the resulting future situations in order to plan their own actions.

In this project, we aim to develop the technology that lays the foundation for applications that require the anticipation of human behavior. Instead of addressing the problem at a limited scope, the project addresses all relevant aspects including time horizons ranging from milliseconds to infinity and granularity ranging from detailed human motion to coarse action labels. To ensure that the developed methods are not limited to a single task but can be applied for a large variety of applications, we do not solve sub-problems in isolation but address the aspects jointly.

As a scenario for an application, we focus on service robots that support impaired or elderly people at home. Due to the demographic change, the population structure in Germany will change dramatically.
Service robots can fill the gap, but they need the ability to anticipate human behavior at various levels of granularity in order to be accepted and be efficient. The robot needs to know when its help is needed, but it should not stand in the way. In a collaborative setting, the robot is expected to complete tasks together with a human. This requires to anticipate both the intention but also detailed movements, e.g., when jointly carrying an object. Another important aspect in this context is the prevention of accidents. This is in particular very important for elderly people. Predicting accidents before they happen would allow to support the humans in time. This can happen by a signal to warn the human if the human can still prevent the accident without additional help, but also by an immediate support of a service robot.


arXiv:2102.05361 (accepted at ACM Journal on Computing and Cultural Heritage (JOCCH)), 2021