Medical imaging techniques are being profoundly used for diagnosis of many diseases. In many of the remote areas in the developing nations, patients who live in rural areas are facing vital health disparities compared to the general population. In such scenarios, eHealth can offer promising solutions. The eHealth especially aims at developing digital applications for offering high accuracy diagnosis even in remote areas. Also, the integration of eHealth with advanced technologies such as deep learning and artificial intelligence will further improve the diagnostic accuracy and also reduces the duration. In this paper, we develop a novel low-cost and easily scalable eHealth architecture comprising of a web application which enables clinicians in fatty liver classification using ultrasound images. The developed web application framework uses a deep learning model (using CNN) for accurate classification of fatty liver using ultrasound images. The clinician in a remote location with a moderate internet connectivity can upload the scanned ultrasound image to the developed web application and the application identifies if any abnormality is present. From the performance analysis, it is observed that the developed model achieves an accuracy of 91.37%. Also, regarding latency the developed classification model predicts the abnormality presence in less than 20 ms. However, including the network latency, it is observed that the developed eHealth architecture predicts with a latency of less than 150 ms using moderate network connectivity. © 2019 IEEE.