Semantic Scene Modeling and Retrieval

by Julia Vogel

In this talk, I will summarize the research done during my PhD at ETH Zurich. The general topic of my dissertation was semantic description, understanding, and modeling of natural scenes.

I will present an image representation that renders it possible to access natural scenes by local semantic description. The basic idea of the semantic modeling is to classify local image regions into semantic concept classes such as water, rocks, or foliage. Images are represented through the frequency of occurrence of these local concepts. Extensive experiments demonstrate that the image representation is well suited for modeling the semantic content of heterogeneous scene categories, and thus for categorization and retrieval.

The image representation also allows us to rank natural scenes according to their semantic similarity relative to certain scene categories. Based on human ranking data, we learn a perceptually plausible distance measure that leads to a high correlation between the human and the automatically obtained typicality ranking.

Time permitting, I will give a short overview of some psychophysical experiments on the importance of global vs. local information in human perception of natural scenes.

Back to the LCI Forum page