Adaptive Systems have been used in a wide range of applications for almost four decades. Examples include adaptive equalization, adaptive noise-cancellation, and adaptive control. The design of adaptive filters/controllers is a difficult nonlinear problem for which good systematic synthesis procedures have proven a difficult challenge. Most commonly-used design methods (e.g. FxLMS) are adhoc in nature and do not provide guaranteed performance level. Systematic analysis of the commonly used adaptive algorithms is also difficult. This work presents an estimation-based synthesis and analysis procedure for adaptive filter design. This work formulates the adaptive filtering/control problem as an H? estimation problem, and updates the adaptive weight vector according to the state estimates provided by an H? estimator. The estimator is proved to be always feasible. Furthermore, the special structure of the problem is used to reduce the usual Riccati recursion for state estimate update to a simpler Lyapunov recursion. The resulting adaptive algorithm has provable performance with simple update rule, and readily extends to multi-channel systems and problems with feedback contamination.