The flow in the atmospheric boundary layer behind an abrupt transition in the surface roughness is studied using large eddy simulations (LES). A key ingredient of the LES is the ‘wall model’ that prescribes the instantaneous shear stress at the bottom surface as a function of the velocity field at the lower-most computational grid point. Two previously-developed wall models are evaluated; the first (BZ) is based on filtering the velocity field and the second (APA) is based on a blending function. The axial evolution of the wall shear stress, and vertical profiles of several quantities such as the stream-wise velocity, turbulence intensity and the total shear stress are evaluated. The APA model is found to predict the shear stress at the surface more accurately than the BZ model when compared with previous experimental results. Not all statistics are found to be equally sensitive to the wall models. We also employ the LES to evaluate an empirical relation for the height of the internal boundary-layer (IBL), which is commonly used to derive analytical models for the mean velocity profiles and wall shear stresses. The empirical relation is found to be reasonably accurate when the IBL height is calculated based on the shear stress but not when it is calculated based on the streamwise velocity variance. These results suggest that the wall model in LES of heterogeneously rough surfaces plays a key role and should be selected carefully in view of the quantity of interest. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.