This paper presents a vision-based motion capture system to quantify changes in temporal domain resulting from motion. The paper also investigates the chaotic behavior of the moving underwater species. The main target is to track the moving object and capture the time sequence behind the moving object. Simulation results show that the proposed algorithm is successful in tracking the moving fishes and in classifying them. The whole task is divided into several modules-image capturing module, image processing module and time series extraction module. The training performance can be improved if it is possible to take larger time series, more training data and faster frame sequence.