PhD defense Maelic Neau: Real-Time And Efficient Scene Graph Generation for Real-World Applications: An End-to-End Investigation

Image Captioning, Visual-Question Answering, or Human-Robot Collaboration are three tasks that rely extensively on a deep understanding of the visual world. To build efficient Artificial Intelligence (AI) algorithms that can solve these tasks, one can build their interpretation of the scene on a graphical structure, what we call a Scene Graph. However, Machine Learning algorithms that generate Scene Graphs are today limited in real-world use cases. In this work, we tackle this challenge by proposing a new method to generate high-quality Scene Graphs datasets, a new inference process to predict more relevant Scene Graphs and a new architecture that uses Scene Graphs for Human-Robot Collaboration. Findings show that our approach can significantly improve the performance of downstream tasks relying on Scene Graphs. At the same time, we also demonstrated the opportunities of using Scene Graphs to understand and plan human-robot collaboration tasks in domestic environments.