Interdisciplinary Institute in Artificial Intelligence

DESCRIPTION

The goal of traditional computer vision is, given an image or a video, to describe the depicted three-dimensional world in terms of objects, scene structure, their physical properties, and spatial relationships. This is motivated by the need for visual representations to enable successful interactions with our environment and others. While this is one important aspect of the story, research in cognitive science has found evidence that, in addition to such visual scene representations, physical contact plays a central role in understanding human-object interactions [1, 2]. In this project, we are interested in learning robot-object interactions and how they emerge from a sequence of contact states with the environment.

The project aims to find answers to the following research questions: How can we learn visual representations informed by an abstract understanding of behavioral strategies for predicting hand-object contact configurations? To what extent does the agent’s environment determine the development of behavioral strategies leading to such hand-object configurations? In this context, some preliminary work shows how high-level understanding of activity supports the learning of future motion trajectories more effectively [3].

The hypothesis motivating this research is that visual representations designed using information derived from interaction will lead to (I) a significant reduction in the amount of training data typically required and (II) better generalization, as these representations are object-agnostic compared to approaches relying on a semantic understanding of objects.

REFERENCES

Zago, M., McIntyre, J., Senot, P., & Lacquaniti, F. (2009). Visuo-motor coordination and internal models for object interception. Experimental Brain Research, 192, 571–604.
Tresilian, J. R. (1995). Perceptual and cognitive processes in time-to-contact estimation: Analysis of prediction-motion and relative judgment tasks. Perception & Psychophysics, 57(2), 231–245
Halawa, M., Hellwich, O., & Bideau, P. (October 2022). Action-based contrastive learning for trajectory prediction. In European Conference on Computer Vision (pp. 143–159). Cham: Springer Nature Switzerland.
 

ACTIVITIES

We currently have PhD positions available for this research chair.

CHAIR EVENTS