The axisymmetric dynamics of a bubble rising in a Bingham fluid under the action of buoyancy is studied. The Volume-of-Fluid (VOF) method is used to solve the equations of mass and momentum conservation, coupled to an equation for the volume fraction of the Bingham fluid. A regularised constitutive model is used for the description of the viscoplastic behavior of the material. The numerical results demonstrate that the rise dynamics are complex for large yield stresses, and for weak surface tension. Under these conditions, for which the bubble is highly deformable, the rise is unsteady and is punctuated by periods of rapid acceleration which separate stages of quasi-steady motion. During the acceleration periods, the bubble aspect ratio exhibits oscillations about unity, whose amplitude and wavelength increase with increasing yield stress and decreasing surface tension. These oscillations are accompanied by the alternating formation and destruction of unyielded zones at the bubble equator, as the bubble appears to "swim" upwards. © 2014 Elsevier B.V.