Although the amount of visual data in the form of digital images and video is increasing at an ever-growing pace, computers are still not capable of automatically analyzing this data on a semantic level. Therefore, users have to process visual input on a low level, rather than enabling an intuitive communication with machines that is based on image content. This book deals with automatic scene understanding and, particularly, with category-level recognition of objects in novel visual scenes, be it images or videos. The research is governed by the main hypothesis that structured, hierarchical object representations can be learned in a fully automatic mode. For this purpose, statistical models are devised that can be learned for each category in an unsupervised manner. This work builds on the fundamental concept of compositionality and it investigates how the compositional nature of our (visual) world can be exploited for simplifying object models.