The assessment of mechanical stiffness is an essential diagnostic tool for investigating the biomechanical properties of biological tissues. Surface wave elastography (SWE) is an emerging technique to quantify elastic properties of tissues in clinical diagnosis. High-speed optical imaging combined with SWE has enormous potential in quantifying the elastic properties of tissues at microscale resolutions. In this study, we implement surface wave elastography using high-speed optical interferometry to characterize the elastic properties of tissue-mimicking phantoms and ex-vivo native caprine liver tissue by imaging the surface wave induced by an electromechanical actuator. The sinusoidal mechanical excitations ranging from 120 Hz to 1.2 kHz on the surface of tissues are captured using a high-speed camera with a frame rate of 4 kHz at micrometer resolutions. The surface wavefront reconstruction is performed using a phase-shifting algorithm and linear regression is used to calculate the surface wave velocity. The mechanical stiffness estimated from the optical system is compared with the results of mechanical compression testing measurements. The results from this multimodal platform combining optical interferometry and vibrational spectroscopy using SWE are highly promising towards a non-invasive or minimally invasive imaging for in-vivo and ex-vivo mechanical characterization of tissues with future clinical applications. © 2022 IOP Publishing Ltd.