In this book, Mean Square Error (MSE) performance of the standard Least Mean Square (LMS), Normalized LMS (NLMS), Leaky LMS, Modified Leaky LMS (MLLMS) and Frequency Response Shaped Least Mean Square (FRS-LMS) algorithms have been investigated in Additive White Gaussian Noise (AWGN) and correlated Gaussian noise environments. The FRS-LMS algorithm has been shown to have superior performance in terms of MSE or speed of convergence compared to the other algorithms. The performance of the FRS-LMS adaptive algorithm in estimating a sinusoidal signal in impulsive and correlated noise is further studied. The algorithm does not require a priori knowledge about the nominal Gaussian process and is able to adapt to changes in the environment. The performance of the FRS-LMS is compared to that of the Leaky-LMS algorithms in terms of MSE and convergence speed. The results indicate that the FRS-LMS provides superior performance in impulsive and correlated noise environments.