Anticipating Human Behavior




In the last years, we have seen a tremendous progress in the capabilities of computer systems to classify image or video clips taken from the Internet or to analyze human pose in real-time for gaming applications. These systems, however, analyze the past or in the case of real-time systems the present with a delay of a few milliseconds. For applications, where a moving system has to react or interact with humans, this is insufficient. For instance, robots collaborating with humans need not only to perceive the current situation, but they need to anticipate human actions and the resulting future situations in order to plan their own actions.

In this project, we aim to develop the technology that lays the foundation for applications that require the anticipation of human behavior. Instead of addressing the problem at a limited scope, the project addresses all relevant aspects including time horizons ranging from milliseconds to infinity and granularity ranging from detailed human motion to coarse action labels. To ensure that the developed methods are not limited to a single task but can be applied for a large variety of applications, we do not solve sub-problems in isolation but address the aspects jointly.

As a scenario for an application, we focus on service robots that support impaired or elderly people at home. Due to the demographic change, the population structure in Germany will change dramatically.
Service robots can fill the gap, but they need the ability to anticipate human behavior at various levels of granularity in order to be accepted and be efficient. The robot needs to know when its help is needed, but it should not stand in the way. In a collaborative setting, the robot is expected to complete tasks together with a human. This requires to anticipate both the intention but also detailed movements, e.g., when jointly carrying an object. Another important aspect in this context is the prevention of accidents. This is in particular very important for elderly people. Predicting accidents before they happen would allow to support the humans in time. This can happen by a signal to warn the human if the human can still prevent the accident without additional help, but also by an immediate support of a service robot.


In: IEEE Computer Graphics and Applications (2021)
Moritz Wolter, Angela Yao, and Sven Behnke
In proceedings of 28th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2020
Moritz Wolter, Juergen Gall, and Angela Yao
In proceedings of International Conference on Artificial Neural Networks, Springer, 2020
Fadime Sener, Dipika Singhania, and Angela Yao
In proceedings of European Conference on Computer Vision (ECCV), 2020
Lilli Bruckschen, Kira Bungert, Moritz Wolter, Stefan Krumpen, Michael Weinmann, Reinhard Klein, and Maren Bennewitz
In proceedings of Proceedings of the IEEE Conference on Robot and Human Interactive Communication (RO-MAN), 2020
Kristina Enes, Hassan Errami, Moritz Wolter, Tim Krake, Bernd Eberhardt, Andreas Weber, and Jörg Zimmermann
In: Sensors (Feb. 2020), 20:4:976
In proceedings of Computer Graphics International Conference, Springer, pages 556-563, 2019
In: Proceedings of the IEEE International Conference on Computer Vision (2019)(862-871)
Julian Tanke, Oh-Hun Kwon, Patrick Stotko, Radu Alexandru Rosu, Michael Weinmann, Hassan Errami, Sven Behnke, Maren Bennewitz, Reinhard Klein, Andreas Weber, Angela Yao, and Juergen Gall
arXiv:1912.06354, Dec. 2019
Patrick Stotko, Stefan Krumpen, Max Schwarz, Christian Lenz, Sven Behnke, Reinhard Klein, and Michael Weinmann
In proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3630-3637, Nov. 2019
In proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 19-25, Oct. 2019
In proceedings of International Conference on 3D Vision (3DV), IEEE, pages 709-718, Sept. 2019
In: Computer Graphics Forum (July 2019), 38:4
In: ISPRS Journal of Photogrammetry and Remote Sensing (May 2019), 151(251-262)
In: IEEE Transactions on Visualization and Computer Graphics (TVCG) (May 2019), 25:5(2102-2112)
In: ISPRS Journal of Photogrammetry and Remote Sensing (P&RS) (Apr. 2019), 150(213-225)
In: Computers & Graphics (Apr. 2019), 79(36-45)
Umar Iqbal, Andreas Doering, Hashim Yasin, Björn Krüger, Andreas Weber, and Juergen Gall
In: Computer Vision and Image Understanding (2018), 172(37-49)
In proceedings of Conference on Neural Information Processing Systems, 2018
In proceedings of Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 2018
Michael Zollhöfer, Patrick Stotko, Andreas Görlitz, Christian Theobalt, Matthias Nießner, Reinhard Klein, and Andreas Kolb
In: Computer Graphics Forum (EG STAR) (May 2018), 37:2(625-652)
Sergey Vakulenko, Ovidiu Radulescu, Ivan Morozov, and Andreas Weber
In: Sensors (2017), 17:12(2907)