Accurate information of path followed by vehicles at several real-life scenarios is essential for modeling and testing the optimal path for autonomous vehicles, analyzing the lateral dynamics such as lane-changing and overtaking, vehicle dynamics at the horizontal curves, intersections, and roundabouts. Recently, the availability of such data is plenty, thanks to the advancements in drone and computer-vision technologies. Nevertheless, extracting useful information from the observed data requires certain pre-processing that can handle the noises added at several stages of the data collection. The adequacy of the existing off-the-shelf smoothing techniques (e.g., Moving average, Savitzky–Golay, Kalman filter) for denoising the trajectory data of the modern era is certainly a question. The motivation for the present study is twofold. Firstly, the challenges of contemporary trajectory data collection and associated error patterns are relatively unknown, and thus an appropriate denoising technique needs to be developed. Secondly, many of the existing smoothing techniques have chosen invariant parameters across all the vehicle paths, even though the error patterns are dissimilar across the vehicles and, in the longitudinal and lateral directions. Thus, an adaptive smoothing needs to be applied for each observed vehicle path and its components. The present study proposes an adaptive, and data-driven path smoothing technique that can be universally applied to the video-based path data obtained from any camera platform (either drone or fixed camera setup). The proposed reconstruction framework works in three-stages; in the first stage, the data is prepared by resampling any missing data. Also, any outlier in the observed path data is identified and removed at this stage. In the second stage, a Recursively Ensembled Low-Pass (RELP) filter is proposed to handle the ’heavily tailed’ noise found in the video-processing based trajectory data, particularly for drone videos. A robust adaptive Gaussian kernel smoothing is applied in the third stage to have a localized reconstruction. For Kernel smoothing, the smoothing parameters such as the optimal bandwidth and polynomial order are estimated using the proposed grid-search algorithm. The parameter estimation process ensures that the bias and variance are perfectly traded-off to achieve a smooth vehicle path. The performance evaluation of the proposed method shows an internally and physically consistent trajectory reconstruction. © 2020 Elsevier Ltd