Tax evasion is most common in several nations. Taxpayers evade tax by using thoughtful and well-considered techniques, which hinders the economic progress of the nation. Delaying the filing of returns by taxpayers is the most primitive form of tax evasion. Taxpayers who delay the filing of returns are called return defaulters. It is the most brazen form of tax evasion. To tackle this problem, we introduce an example-dependent cost-sensitive deep learning model to identify potential return defaulters. This model takes example-dependent costs into account and makes predictions that aim to minimize the overall cost instead of minimizing the total number of misclassifications. Applying our method, we show cost savings of about 55%. This work is designed and implemented for the Commercial Taxes Department Government of Telangana, India. © 2020 IEEE.