Echocardiography is a widely used and cost-effective medical imaging procedure that is used to diagnose cardiac irregularities. To capture the various chambers of the heart, echocardiography videos are captured from different angles called views to generate standard images/videos. Automatic classification of these views allows for faster diagnosis and analysis. In this work, we propose a representation for echo videos which encapsulates the motion profile of various chambers and valves that helps effective view classification. This variety of motion profiles is captured in a large Gaussian mixture model called universal motion profile model (UMPM). In order to extract only the relevant motion profiles for each view, a factor analysis based decomposition is applied to the means of the UMPM. This results in a low-dimensional representation called motion profile vector (MPV) which captures the distinctive motion signature for a particular view. To evaluate MPVs, a dataset called ECHO 1.0 is introduced which contains around 637 video clips of the four major views: a) parasternal long-axis view (PLAX), b) parasternal short-axis (PSAX), c) apical four-chamber view (A4C), and d) apical two-chamber view (A2C). We demonstrate the efficacy of motion profile-vectors over other spatio-temporal representations. Further, motion profile-vectors can classify even poorly captured videos with high accuracy which shows the robustness of the proposed representation. © 1982-2012 IEEE.