Supervised classification and data clustering are two fundamental disciplines of data mining and machine learning where probabilistic graphical models, and particularly Bayesian networks, have become very popular paradigms. This book aims to contribute to the state of the art of both supervised classification and data clustering disciplines by providing new algorithms to learn Bayesian networks. On the one hand, the contributions related to supervised classification are focused on the discriminative learning of Bayesian network classifiers. Part of this book tries to motivate the use of this discriminative approach and presents new proposals to learn both structure and parameters of Bayesian network classifiers from a discriminative point of view. On the other hand, the part related to data clustering introduces new methods to deal with Bayesian model averaging for clustering. Additionally, the proposed methods are evaluated in diferent sinthetic and real datasets including a real problem taken from the field of population genetics.