Single-shot reconstruction of the inline hologram is highly desirable as a cost-effective and portable imaging modality in resource-constrained environments. However, the twin image artifacts, caused by the propagation of the conjugated wavefront with missing phase information, contaminate the reconstruction. Existing end-to-end deep learning-based methods require massive training data pairs with environmental and system stability, which is very difficult to achieve. Recently proposed deep image prior (DIP) integrates the physical model of hologram formation into deep neural networks without any prior training requirement. However, the process of fitting the model output to a single measured hologram results in the fitting of interference-related noise. To overcome this problem, we have implemented an untrained deep neural network powered with explicit regularization by denoising (RED), which removes twin images and noise in reconstruction. Our work demonstrates the use of alternating directions of multipliers method (ADMM) to combine DIP and RED into a robust single-shot phase recovery process. The use of ADMM, which is based on the variable splitting approach, made it possible to plug and play different denoisers without the need of explicit differentiation. Experimental results show that the sparsity-promoting denoisers give better results over DIP in terms of phase signal-to-noise ratio (SNR). Considering the computational complexities, we conclude that the total variation denoiser is more appropriate for hologram reconstruction. © 2023 Author(s).