TU Bergakademie Freiberg - Institute for Informatics - Virtual Reality and Multimedia Group

  Virtual Workers Virtual Workers

Virtual Workers

Interactive Digital Humans in Virtual Prototyping



Duration
since 2005 (follow-up research to the Virtuelle Werkstatt project)


Description
The project ´Virtual Workers´ aims to develop novel methods in Virtual Reality (VR) based virtual prototyping through the inclusion of semi-autonomous, intelligent digital humans. A main goal concerns the transmission of operating, maintenance, and assembly procedures performed by a human VR user to digital humans of different body shapes and sizes. In this way, not only the effort required for the generation of 3D animations will be greatly reduced; furthermore, functional and ergonomic verification of virtual prototypes can be accomplished in a generalized manner. The following individual goals are pursued:
  • From digital avatars to autonomous, AI-enhanced virtual workers: Computer graphical digital humans will be extended with simulated sensors and an Artificial Intelligence-based control architecture with deliberative and reactive layers. The resulting ´virtual workers´ will be capable of manipulating the virtual prototypes in an autonomous fashion.
  • From virtual prototype evaluation by one VR user to the evaluation by many virtual workers: The virtual workers will learn to perform operating, maintenance, and assembly procedures by imitation of human VR users. The learnt procedures are stored as high-level plans that can be adapted on-the-fly to virtual humans of different sizes.
  • Implementation of an integrated VR system supporting the design, evaluation, and variation of virtual prototypes: Operation, assembly, and other procedures are performed either interactively by the human VR user, or - through the application of imitation learning techniques - by virtual workers of different sizes.

 
See also
The action capture page provides more information on an VR-based approach to imitation learning, by which the virtual workers learn to repeat interactively demonstrated procedures on virtual prototypes.
Our work on grasp recognition compares a variety of classification methods for hand shape recognition during grasping. It shows, that a reliable classification of grasp types can be achieved even with uncalibrated data gloves.

Team
Prof. Dr.-Ing. Bernhard Jung
Dipl.-Inform. Guido Heumer
Dipl.-Inform. Arnd Vitzthum
Dipl.-Inform. Heni Ben Amor
Maik Deininger
Frank Gommlich
Peter Menzel
Frank Winkler
Media Images and videos

Publications
G. Heumer, H. Ben Amor & B. Jung. Grasp Recognition for Uncalibrated Data Gloves - A Machine Learning Approach. Presence - Teleoperators and Virtual Environments, vol 17(2), April 2008. MIT Press.

G. Heumer, H. Ben Amor, M. Weber & B. Jung. Grasp Recognition with Uncalibrated Data Gloves – A Comparison of Classification Methods. Proceedings IEEE Virtual Reality 2007, pp. 19 - 26. (*Honorable Mention) PDF

H. Ben Amor, M. Weber, G. Heumer & B. Jung. Coordinate System Transformations for Imitation of Goal-Directed Trajectories in Virtual Humans. In Virtual Environments 2007. IPT EGVE 2007. 13th Eurographics Symposium on Virtual Environments. Short Papers and Posters. 2007. 45-46.

M. Weber, G. Heumer, H. Ben Amor & B. Jung. An Animation System for Imitation of Object Grasping in Virtual Reality. Proceedings ICAT 2006 - 16th International Conference on Artificial Reality and Teleexistence. 2006. PDF

B. Jung, H. Ben Amor, G. Heumer & M. Weber. From Motion Capture to Action Capture: A Review of Imitation Learning Techniques and their Application to VR-based Character Animation. Proceedings VRST 2006 – Thirteenth ACM Symposium on Virtual Reality Software and Technology, 2006, pp. 145-154.  PDF

H. Ben Amor, S. Ikemoto, T. Minato, B. Jung & H. Ishiguro. A Neural Framework for Robot Motor Learning based on Memory Consolidation. ICANNGA-07 – International Conference on Adaptive and Natural Computing Algorithms. 2007, pp. 461-468.   (*Best Young Researcher Paper Award)   PDF

G. Heumer, M. Weber, H. Ben Amor & B. Jung. Calibration-free Recognition of Grasp Types – A Comparison of Classification Methods. Proceedings Dritter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR. 2006.

H. Ben Amor, S. Ikemoto, T. Minato & H. Ishiguro. Learning Android Control using Growing Neural Networks. Proceedings of JSME Robotics and Mechatronics Conference ROBOMEC, 2006.

M. Weber, G. Heumer & B. Jung: Towards Grasp Learning in Virtual Humans by Imitation of Virtual Reality Users. In Processings Zweiter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR. Shaker-Verlag. 2005.  PDF

M. Weber, T. Pfeiffer & B. Jung: Pr@senZ - P@CE: Mobile Interaction with Virtual Reality. In Proceedings Mobile HCI - 7th International Conference on Human Computer Interaction with Mobile Devices and Services. 2005. PDF

T. Pfeiffer, M. Weber & B. Jung: Ubiquitous Virtual Reality - Accessing Shared Virtual Environments through Videoconferencing Technology. Proceedings Theory and Practice of Computer Graphics Conference. Eurographics, 2005, pp. 209-216. (*Ken Brodlie Price for Best Paper) PDF

Funding Research in the Virtual Workers is funded by the Deutsche Forschungsgemeinschaft (DFG)


March 2008