The objective of this paper is to demonstrate the usefulness of phase derived from the linear prediction (LP) residual for speaker recognition. Though the sequence of samples in the LP residual are uncorrelated, they are not independent. Since the magnitude spectrum of the LP residual is almost flat, the dependencies among the samples in LP residual are reflected mainly in its phase spectrum. The information in the phase spectrum of the LP residual is captured by modeling LP residual as the output of an allpass filter excited by independent and identically distributed (i.i.d.) nongaussian input. The coefficients of the allpass filter are estimated iteratively using higher order cumulants of the input. The estimated coefficients are used as features to build a speaker recognition system using Gaussian mixture models. The speaker recognition system built from the proposed features resulted in an equal error rate of 6% on a population of 50 speakers. © 2012 IEEE.