Purpose: To evaluate the effect of sub-mucosal mechanical instrumentation following supra-mucosal plaque removal in the treatment of peri-implant mucositis. Materials and methods: A total of 44 patients, 27 from a specialist clinic in Sweden, and 17 from a Japanese one, were included. The 44 patients had a total of 175 implants of which 84 (48%) had clinical signs of mucositis (bleeding on probing [BoP]) and no radiographic bone loss. At baseline, the 44 patients received professional supra-mucosal plaque removal and oral hygiene instruction and motivation. Following the 1-month examination, sub-mucosal instrumentation was initiated using ultrasonic and hand instruments. Patients were then examined at 4 and 7 months. Presence of plaque, marginal bleeding, BoP and probing pocket depth (PPD) changes were recorded at each examination in an unblinded manner. Results: One month following supra-mucosal plaque removal, the number of treated implants with BoP was reduced from 84 to 44 (48% reduction) with a concomitant decrease of the mean PPD of 0.6 mm (95% CI: −0.5 to −0.7), from 4.4 ± 1.0 mm (mean ± SD) to 3.8 ± 1.0 mm, P < 0.0001. Following sub-mucosal instrumentation, a further reduction of 9% (from 44% to 36%) of BoP was recorded, with a concomitant reduction of the mean PPD of 0.3 mm (95% CI: −0.1 to −0.4) at the 7-month examination. Conclusions: The improvement of the clinical condition (absence of marginal bleeding, BoP and PPD reduction at treated implants) following non-surgical treatment of mucositis appeared to be in great part because of supra-mucosal plaque removal, while the sub-mucosal instrumentation seemed to have only a minor additional effect. Deep implant position compared to neighbouring teeth/implant was associated with persisting mucositis following treatment. The presence of deep pockets following treatment was associated with BoP, screw-retained prostheses without abutment and sub-mucosal crown margins.