The task of segmentation and tracking of objects in a video sequence is an important high-level video processing problem for object-based video manipulation and representation. This task involves utilization of many low-level preprocessing tasks such as image segmentation and motion estimation. It is also very important to assess the performance of the video object segmentation and tracking algorithms quantitatively and objectively. In this book, performance evaluation measures are proposed both when the ground-truth segmentation maps are available and when they are unavailable. An active contour based video object tracking method is introduced that uses the proposed performance evaluation measures in a feedback loop to adjust its parameters locally on the object boundary. This book also provides two novel region-based methods for video object tracking using active contours based on histogram and de-focus energies. New low-level image segmentation and motion estimation algorithms, namely, an illumination invariant fuzzy image segmentation algorithm and a motion estimation estimation algorithm in the frequency domain using fuzzy c-planes clustering are also presented in this book.