Convolutional Neural Networks also known as ConvNets are now extensively used in various machine learning tasks for solving various problems in computer vision, biomedical, defence industry, entertainment etc. These neural networks for most of the applications are focused towards increasing the accuracy. However, besides maintaining the accuracy within a tolerable range, reduction in the network model size can have a lot of advantages from its mobility, easy deployment, remote upgradation and energy efficiency point of view. To attain these advantages, we propose a universal strategy to realize the convolution operation of a n x n filter kernel with fewer parameters, which also reduces the number of channels. We have proposed a compressed VGGNet model based on VGGNet neural network which resulted in 20x lesser parameters compared to its classical counterpart with an improved inference time by 3 times whilst maintaining similar accuracy. A complete hardware design of the compressed VGG architecture has also been implemented. A quantitative and qualitative analysis for various variants of VGGNet and other models reveal the reduction in the number of parameters in the range of 18-20x and the number of network operations contributing to the model complexity has shown a reduction of 2.5x with respect to its vanilla counterpart making it easier to deploy onto FPGAs and edge devices © 2022 IEEE.