We propose a full reference stereo video quality assessment algorithm for assessing the perceptual quality of natural stereo videos. We exploit the separable representation of motion and binocular disparity in the visual cortex and develop a four stage algorithm to measure the quality of a stereoscopic video called FLOSIM3D. First, we compute the temporal features by utilizing an existing 2D VQA metric which measures the temporal annoyance based on patch level statistics such as mean, variance and minimum eigen value and pools them with a frame categorization based non-linear pooling strategy. Second, a structure based 2D Image Quality Assessment (IQA) metric is used to compute the spatial quality of the frames. Next, the loss in depth cues is measured using a structure based metric. Finally, the features for each of the stereo views are pooled to obtain the final stereo video quality score. We demonstrate the state-of-the-art performance of the proposed metric on IRCCYN dataset involving H.264, JP2K compression artifacts. © 2017 IEEE.