Achieving an optimal design of phase-locked loop (PLL) is a major challenge in WiMAX technology in order to improve system behavior against noise and to enhance Quality of Service (QoS). A new loop filter design method for PLL is introduced taking into consideration various design objectives such as small settling time, small overshoot and meeting Fixed WiMAX requirements. Optimizing conflicting objectives is accomplished using semi-Definite programming (especially Linear Matrix Inequality (LMI)) in conjunction with appropriate adjustment of certain design parameters. Infinite Impulse Response (IIR) and Finite Impulse Response (FIR), Digital filters are designed by Semi-Definite Programming (SDP) using SeDuMi (self-dual minimization) toolbox software. Design efficiency and performance of the proposed method are illustrated by simulations and comparisons to other design methods. Simulations showed that the IIR which was designed by SDP is better than the IIR which designed by Linear Programming (LP).Also, the FIR which was designed by SDP using SeDuMi software is better than the FIR which was designed in by SDP using CVX (convex optimization tool) software.