Representing knowledge that facilitate learning, planning and decision making in complex, stochastic and uncertain environments is a critical issue for many application domains. The monograph consists of a wealth of new theoretical and advanced fundamental techniques for approximate decision making under uncertainty and for knowledge representation in partially observable systems. The framework enables decision making agents to reason under uncertain conditions and learn without direct supervision, from interaction with their environment.