Nonlinear dynamics play an important role on biological livings, including biological neural systems. Researchers have shown that nonlinear dynamics and chaos exist in several levels of neural systems from micro to macro levels. This is a strong motivation for the addition of nonlinear dynamics and chaos into neural networks. In this book, a new type of neural network is introduced based on the nonlinear dynamics and chaos. Bifurcation behavior of the neural network shows that the state of the neurons will move chaotically among the stored states of the network but finally, by decreasing the degree of chaos, settles down into a proper state. Therefore this new type of neural network with chaotic dynamics is able to search the state space and seeks the points far away from equilibrium points. We also find that increasing the memory capacity of the neural network is one of the consequences of adding the nonlinearity into it. This network can also be used for pattern recognition, classification, and optimization.