Computer Science professor Derek Hoiem’s paper “Indoor Segmentation and Support Inference from RGBD Images,” published at the 12th European Conference on Computer Vision, Florence, Italy in 2012, has won the 2022 Koenderink Prize, a test of time award for impactful papers. The Koenderink Prize “recognizes fundamental contributions in computer vision.” This prize is awarded at ECCV each year for a paper published ten years ago at that conference.
The research presented in Hoiem’s paper focused on interpreting major surfaces, objects, and support relations in indoor scenes from RGBD (red, green, blue plus depth) images. Prior work often focused on tidy rooms that lack clutter and physical interactions. The goal of this research was to parse regular (often messy) indoor scenes and categorize portions of the image into surfaces and objects and to infer support relations. After capturing all the images, Hoiem said “we wanted to label every single thing that’s in all of the images and to have freeform labels. Even if there’s a cluttered desk that might have 20 objects on it, that all got labeled.” The purpose of annotating all of the objects in each image was to infer support relations in complex scenes. Support relations refers to the physical relationships between objects, possible actions that can be performed, or the geometric structure of the scene.
The authors ended up with around 500 different scenes and about three images per scene. One of the challenges they faced was the annotation effort of creating very clean annotations of all the different objects in all of the images. This large quantity of images became a unique dataset that covered a lot of spaces where people actually live and work, had depth in RBG images, and had a huge diversity of objects. This dataset has become widely used in the computer vision area, as well as for robotics.
Hoiem said that the biggest impact of the work has been influencing and enabling other research.
“The paper has been cited over 4,600 times and a lot of those citations are not just people citing that it’s [image dataset] there, but they’re using the data for evaluation or for development,” he added.
Reflecting on the longevity and impact of this paper, Hoiem said, “sometimes it can feel like you’re in a sea of research and it’s hard to see what impact you’re having when there’s so many papers coming out every day. But it’s nice when you can look back and see something that really influenced that field and helps to drive research in a way that is good for the research community and leads to interesting applications.”