RoboCup soccer, an education initiative and a solid platform for Machine Learning and robotics international research. RoboCup, provides a high dimensional state and action space multi-agent problem, in which autonomous agents have to work collectively to achieve a common goal; (in this case) with their teammates (other robots) to win the match. Few has been written about RoboCup Soccer that provides a full framework about practical implementation of Reinforcement learning algorithms in this field, being this the first purpose of this book, take the reader to applied artificial intelligence. Also this book compares two Reinforcement Learning algorithms applied to learning RoboCup skills, two individual skills (Move to position, Intercept) and a multi-agent skill: Keepaway, proposed by Peter Stone in 2D, then scaling this skill to the 3D server, providing the results of which algorithm is better for each kind of problem. The research and results presented in this book cover remarkable topics for researchers, students and anyone interested in Reinforcement Learning or anyone else involved or participating in RoboCup soccer world cup.