Purpose: Ultrasound scanning has been widely used for preliminary diagnosis as it is non-invasive and has good scope for the doctors to analyze many diseases. Due to lack of trained radiologists in remote areas, tele-radiology is used to diagnose the scanned ultrasound data. Availability of online radiographers and having communication facility for the portable ultrasound are issues in tele-radiology for using ultrasound scanning in remote health-care. In these situations, Computer Aided Diagnosis (CAD) will be beneficial in diagnosing the patients with minimal manual intervention. Methods: We proposed FPGA based CAD algorithm for abnormality detection of kidney in ultrasound images. The proposed algorithm works in the following way: as a pre-processing, an ultrasound image is denoised and region of interest of kidney in ultrasound image is segmented. Intensity histogram features and Haralick features are extracted from the segmented kidney region. Based on extracted features, the classification algorithm is implemented in two stages. In first stage, a Look Up Table (LUT) based approach is used to differentiate between normal and abnormal kidney images. In second stage, after confirming the abnormality, Support Vector Machine (SVM) with Multi-Layer Perceptron (MLP) classifier trained with extracted features is used to further classify the presence of stone or cyst in kidney. The proposed algorithm is implemented on a FPGA based Xilinx Kintex-7 board. Results: The proposed algorithm gave an accuracy of 98.14%, sensitivity of 100% and specificity of 96.82% in detecting the exact abnormality present in kidney ultrasound images. Conclusion: The proposed algorithm and its hardware implementation will be beneficial for diagnosing the kidney in absence of radiologists and internet connectivity. © 2016 AGBM