The field of music and speech classification is quite mature with researchers having settled on the approximate best discriminative representation. In this regard, Zubair et al. showed the use of sparse coefficients alongwith SVM to classify audio signals as music or speech to get a near-perfect classification. In the proposed method, we go one step further, instead of using the sparse coefficients with another classifier they are directly used in a dictionary which is learned using on-line dictionary learning for music-speech classification. This approach removes the redundancy of using a separate classifier but also produces complete discrimination of music and speech on the GTZAN music/speech dataset. Moreover, instead of the high-dimensional feature vector space which inherently leads to high computation time and complicated decision boundary calculation on the part of SVM, the restricted dictionary size with limited computation serves the same purpose. © 2014 IEEE.